How can I set up logging for aiohttp.client when making a request using aiohttp.ClientSession ()?

I have some code making a sequence of requests to some API. I would like to set up a common log for everyone, how can I set this up?

Let's say my code looks like this:

import aiohttp
import asyncio

async def fetch(client):
    async with client.get('http://httpbin.org/get') as resp:
        assert resp.status == 200
        return await resp.text()

async def post_data(client):
    async with client.post('http://httpbin.org/post', data={'foo': 'bar'}) as resp:
        assert resp.status == 200
        return await resp.text()

async def main(loop):
    async with aiohttp.ClientSession(loop=loop) as client:
        html = await fetch(client)
        print(html)
        other_html = await post_data(client)
        print(other_html)

loop = asyncio.get_event_loop()
loop.run_until_complete(main(loop))

      

Now I would like to see the status code, url, headers and everything for all requests made, so the output in the logs looks like this:

2017-08-09 08:44:30 DEBUG (200) <GET http://httpbin.org/get>
2017-08-09 08:44:30 DEBUG (200) <POST http://httpbin.org/post>

      

I know that after each request a call to logger.log () can be added, but this will be duplicate. If I have more requests, I will have to write duplicate code for every request that calls logger.log. Seems ineffective.

There is a log aiohttp.client , but no details on how to set it up.

I am trying to set it up this way

logger = logging.getLogger('simple_example')
logger.setLevel(logging.DEBUG)
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
ch.setFormatter(formatter)
logger.addHandler(ch)
logging.getLogger('aiohttp.client').addHandler(ch)

      

but it doesn't print the information I would like to see (e.g. response status code, url).

Is there a way to achieve what I need? Perhaps I can subscribe to some signal from the client and record a message when the signal is sent? For example. is there some mechanism to subscribe to a signal sent when a client receives a response and then logs a message to that?

+3


source to share


1 answer


As you can see in the aiohttp code, the aiohttp.client logger is not used to log requests, but can only log a warning if the cookies in the response are invalid https://github.com/aio-libs/aiohttp/search?utf8=%E2 % 9C% 93 & q = client_logger & type =

To log all your requests, you need to create a custom ClientSession

one that does what you want. Something like:

class LoggingClientSession(aiohttp.ClientSession):
    def request(self, method, url, **kwargs):
        logger.debug('Starting request <%s %r>', method, url)
        return super().request(method, url, **kwargs)

      

-



Jaanus noted in the comments here post

, get

... assistants know the call ClientSession._request

directly instead request

. This way, the override will later not intercept calls made with short helpers.

So you can:

  • override _request

    instead request

    in your helper

  • or make sure your code never uses get

    / ... helpers and always calls request

    directly.

  • and also define all helper methods in LoggingClientSession

+6


source







All Articles