The concurrency task is much slower than using System.Threading.Thread directly
I have used System.Threading.Task and System.Net.Http.HttpClient to upload a test to a web server and observed some strange behavior. Here is the code:
var taskArray = new List<Task>();
for (int i = 0; i < 105; i++)
taskArray.Add(Task.Factory.StartNew(() => Get("/content/test.jpg")));
Task.WaitAll(taskArray.ToArray());
Even though each request (when checked through the fiddler) took only about 15ms to complete, I was getting timeout exceptions (well, TaskCanceledExcpetions are the same thing) that HttpClient used to request when the request timed out the default timeout 100 seconds.
The first thing I tried was increasing the timeout on the HttpClient, which worked, but I was still trying to figure out why requests with very short response times were timings. So I set a timer before I called HttpClient.PostAsync and checked how long it took to complete, the time was suspected to be over 100 seconds, even though the server sent the response much faster.
Then I read that HttpClient.Timeout is a timeout for the entire async operation, which made me think that maybe the task scheduler was causing me problems by only executing the async callback that got the response when after the response was ready to be received.
With this in mind, I decided to write code using good old System.Threading.Thread:
var handle = new EventWaitHandle(false, EventResetMode.ManualReset);
for (int i = 0; i < 105; i++)
{
var t = new Thread(() =>
{
Get("/content/test.jpg");
if (Interlocked.Decrement(ref numberOfTasks) == 0)
handle.Set();
});
t.Start();
}
handle.WaitOne();
This works as expected! I can even scroll the number of threads up to 2000 and complete them all faster than the task based version took 105 to send!
What gives?
source to share
As Matthew mentioned, this is because the Task factory uses the default thread pool. If you limit a little or create your own threadpool, you can use some value if you want to use tasks and want to increase the thread limit.
However, you are already working with a class that has all the async methods: why are you trying to wrap them in streams? Just complete the tasks and wait for them.
Something like that:
var tasks = new List<Task>();
for (int i = 0; i < 105; i++)
tasks.Add(client.GetAsync(uri));
Task.WaitAll(tasks.ToArray());
source to share