What is the best way to make multiple requests to an API?

Question

What is the best way to make multiple requests to an API?

Navigation

#1 by (3 votes)

4

I came to put this question one more time thinking that it would be beneficial for the community to have a canonical way of making multiple http requests to a particular API.

More specifically this question motivated me to put my question.

What is the best way to make multiple requests to an API? (Consider reducing the time needed to get all the answers as much as possible)

To be a bit more concrete: How can I get the first 10 pages of the stackechange api, link the first page.

http c# paralelismo

asked by anonymous 19.05.2016 / 11:09

1 answer

How to make a profile graph in R? How to customize a Select in columns?

score 3 · Accepted Answer

Using Tasks

The idea is to make requests asynchronously and add them to a to-do list with all requests. Then we wait for the respsotas with Task.WaitAll :

var tasks = new Task<string>[10];
var api = "https://api.stackexchange.com/2.2/answers?page={0}&pagesize=10&site=stackoverflow";
using(var client = new HttpClient()){
    for(int i = 1 ; i < 11; ++i){
        var url = string.Format(api, i);
        tasks[i-1] = client.GetAsync(url)
            .ContinueWith(t => t.Result.Content.ReadAsStringAsync())
            .Unwrap();
    }

    Task.WaitAll(tasks);
    foreach (var resposta in tasks){
        Console.WriteLine(resposta.Result);
    }
}

Actually this other question questions this same approach.

An alternative (according to a quick measurement) that is more efficient than above and possibly more understandable can be obtained by using the keyword await :

var tasks = new List<Task<HttpResponseMessage>>();
using (var client = new HttpClient())
{
    for (int i = 1; i < paginas; ++i)
    {
        var url = string.Format(api, i);
        tasks.Add(client.GetAsync(url));
    }

    var respostas = await Task.WhenAll(tasks);
    var todas = new List<string>();
    foreach(var resposta in respostas)
    {
        todas.Add(await resposta.Content.ReadAsStringAsync());
    }
    return todas;
}

But never make the following code:

var respostas = new List<string>();
using (var client = new HttpClient())
{
    for (int i = 1; i < paginas; ++i)
    {
        var url = string.Format(api, i);
        var resposta = await client.GetAsync(url);
        respostas.Add(await resposta.Content.ReadAsStringAsync());
    }
}
return respostas;

With this code requests are not being made in parallel and take longer.