Making more than 1000 requests HTTP requests fail
I am trying to load an API test. I am running tasks concurrently, each doing an HTTP request. I use Task.WhenAll(mytasks)
to wait for all tasks to complete. The requests look like this:
using (var response = await client.SendAsync(request).ConfigureAwait(false))
{
using (var jsonResponse = await response.Content.ReadAsStreamAsync().ConfigureAwait(false))
{
var jsonSerializer = new DataContractJsonSerializer(typeof(Borders));
var borders = (Borders)jsonSerializer.ReadObject(jsonResponse);
return borders;
}
}
This works for at least a thousand tasks. However, if I ran several thousand tasks, I ran into HttpRequestExceptions
:
System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
--- End of inner exception stack trace ---
at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar)
--- End of inner exception stack trace ---
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
--- End of inner exception stack trace ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at BusinessLogic.<GetBorder>d__6d.MoveNext() in c:\BusinessLogic.cs:line 757
So my questions are: why is this happening (with over 1000 jobs)? How can I prevent this? I could obviously cut my block of tasks into chunks <1000, but I would like to leave that in the base system ...
source to share
I would like to leave this in the base system ...
This is not a good idea .. The NET Framework has zero competence in determining the optimal degree of parallelism for IO.
It is generally not advisable to issue as many queries in parallel because the resources that are underlined here are likely to exceed this. Apparently your backend server is not designed to handle this degree of parallelism. It forcibly disconnects you according to the message.
Just because we have simple asynchronous I / O from await
now doesn't mean you can spam your resources with 1000 concurrent requests.
Use one of the general solutions to perform a series of asynchronous actions with the degree of parallelism set. I like http://blogs.msdn.com/b/pfxteam/archive/2012/03/05/10278165.aspx but there are solutions based on ActionBlock
.
source to share
As stated in his answer, your server is closing connections, resulting in an error message:
An existing connection was forcibly closed by the remote host.
You still don't know what exactly your bottleneck is. It may be that your server is unable to process requests, it might be a server blocking them due to some rate limiting function, or it might also be a stress client due to the fact that it cannot process responses fast enough and therefore too much connections are open. You will need to collect additional information before you can optimize your service.
Instead of writing your own stress testing tool, I would recommend using an installed one like Apache JMeter . You should be able to build a test plan that resembles the test script you want to run.
At some point you will reach a user count that one machine cannot simulate. There are services (like Redline13 ) that allow you to run tests from EC2 instances and provide you with tools to analyze the resulting data. You can also use this JMeter EC2 script to execute a test plan from multiple computers.
source to share
I wrote this gist a while ago
//PM> Install-Package Rx-Linq
readonly List<string> _list = new List<string> { "http://www.google.com", "https://www.gmail.com", "http://www.aripaev.ee" };
private readonly string format = "[{0}] {1} : {2} [{3}]";
[Category("WebClient")]
[TestCase("sync" )]
public void SynchronousTest(string name)
{
DateTime start = DateTime.Now;
var dict = _list.ToDictionary(o => o, o => new WebClient().DownloadString(new Uri(o)));
dict.Keys.ToList().ForEach(o => Console.WriteLine(format, DateTime.Now - start, o, dict[o].Length, name));
}
[Category("WebClient")]
[TestCase("async")]
public void AsynchronousTest(string name)
{
DateTime start = DateTime.Now;
var dict = _list.ToDictionary(o => o, async o => await new WebClient().DownloadStringTaskAsync(new Uri(o)));
dict.Keys.ToList().ForEach(o => Console.WriteLine(format, DateTime.Now - start, o, dict[o].Result.Length, name));
}
[Category("WebClient")]
[TestCase("lazy")]
public void LazyTest(string name)
{
var start = DateTime.Now;
var dict = _list.ToDictionary(o => o, o => new Lazy<string>(() => new WebClient().DownloadString(new Uri(o))));
dict.Keys.ToList().ForEach(o => Console.WriteLine(format, DateTime.Now - start, o, dict[o].Value.Length, name));
}
With the following results:
[00:00:00.9520952] http://www.google.com : 51348 [sync]
[00:00:00.9520952] https://www.gmail.com : 58704 [sync]
[00:00:00.9520952] http://www.aripaev.ee : 208324 [sync]
[00:00:00.0010001] http://www.google.com : 51412 [lazy]
[00:00:00.6550655] https://www.gmail.com : 58664 [lazy]
[00:00:00.7690769] http://www.aripaev.ee : 208324 [lazy]
[00:00:00.1430143] http://www.google.com : 51366 [async]
[00:00:00.3430343] https://www.gmail.com : 58616 [async]
[00:00:00.5150515] http://www.aripaev.ee : 208324 [async]
A single machine / server can handle 300-500 concurrent requests, but even this is a stress test for system / network resources.
source to share