Error in requests with aiohttp in asyncio

Question

Error in requests with aiohttp in asyncio

Navigation

#1 by (1 votes)

0

import time
import urllib.request
import asyncio
import aiohttp

URL = 'https://api.github.com/events'
MAX_CLIENTS = 10


def fetch_sync(pid):
    print('Fetch sync process {} started'.format(pid))
    start = time.time()
    response = urllib.request.urlopen(URL)
    datetime = response.getheader('Date')

    print('Process {}: {}, took: {:.2f} seconds'.format(
        pid, datetime, time.time() - start))

    return datetime


async def fetch_async(pid):
    print('Fetch async process {} started'.format(pid))
    start = time.time()
    response = await aiohttp.request('GET', URL)
    datetime = response.headers.get('Date')

    print('Process {}: {}, took: {:.2f} seconds'.format(
        pid, datetime, time.time() - start))

    response.close()
    return datetime


def synchronous():
    start = time.time()
    for i in range(1, MAX_CLIENTS + 1):
        fetch_sync(i)
    print("Process took: {:.2f} seconds".format(time.time() - start))


async def asynchronous():
    start = time.time()
    tasks = [asyncio.ensure_future(
        fetch_async(i)) for i in range(1, MAX_CLIENTS + 1)]
    await asyncio.wait(tasks)
    print("Process took: {:.2f} seconds".format(time.time() - start))


print('Synchronous:')
synchronous()

print('Asynchronous:')
ioloop = asyncio.get_event_loop()
ioloop.run_until_complete(asynchronous())
ioloop.close()

I'm studying the asyncIO module of Python and I came across that code in GitHub ( link ) but the same is generating this error:

Traceback (most recent call last):
  File "test_one.py", line 198, in fetch_async
    response = await aiohttp.request('GET', URL)
TypeError: object _SessionRequestContextManager can't be used in 'await' expression

I do not know much about the module yet, could someone tell me why this error is occurring and how can I do it correctly. Thank you in advance.

python assincronismo python-3.x

asked by anonymous 12.04.2018 / 07:19

1 answer

Separate Data from Multiple Forms with FormData PostgreSQL query bringing more results than expected

score 1 · Accepted Answer

There is a much simpler way to resolve this. First, let's define an asynchronous function that will be responsible for making the request, using the module aiohttp :

async def fetch(url):
  async with aiohttp.ClientSession() as session:
    async with session.get(url) as response:
      return await response.json()

The function basically defines a session, performs a GET request for the specified URL, and returns the response decoded as JSON. Based on this function, we define the function that will handle the obtained answers:

async def get(url, name):
  print(f'Iniciando a requisição {name}')
  response = await fetch(url)
  print(f'Quantidade de eventos da requisição {name}:', len(response))

The function basically performs the request, waits for the response, and displays the number of records that it obtained. The magic here happens in await , because during the HTTP request, the Python interpreter does not have to be locked into this routine, and can be directed to the others, making all the requests in parallel. Thus, it is worth remembering that the order that the requisitions will be made is not guaranteed.

To use the get function, we can do:

requests = [get(URL, i) for i in range(MAX_CLIENTS)]

And we must create the event loop to handle all requests:

loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(*requests))

The final code thus becomes:

import aiohttp
import asyncio

URL = 'https://api.github.com/events'
MAX_CLIENTS = 10

async def fetch(url):
  async with aiohttp.ClientSession() as session:
    async with session.get(url) as response:
      return await response.json()

async def get(url, name):
  print(f'Iniciando a requisição {name}')
  response = await fetch(url)
  print(f'Quantidade de eventos da requisição {name}:', len(response))

requests = [get(URL, i) for i in range(MAX_CLIENTS)]

loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(*requests))

See working at Repl.it

One possible output of the program is:

Iniciando a requisição 1
Iniciando a requisição 7
Iniciando a requisição 2
Iniciando a requisição 8
Iniciando a requisição 3
Iniciando a requisição 9
Iniciando a requisição 4
Iniciando a requisição 5
Iniciando a requisição 0
Iniciando a requisição 6
Quantidade de eventos da requisição 8: 30
Quantidade de eventos da requisição 2: 30
Quantidade de eventos da requisição 0: 30
Quantidade de eventos da requisição 6: 30
Quantidade de eventos da requisição 5: 30
Quantidade de eventos da requisição 1: 30
Quantidade de eventos da requisição 9: 30
Quantidade de eventos da requisição 3: 30
Quantidade de eventos da requisição 4: 30
Quantidade de eventos da requisição 7: 30