How to make this loop body parallel in C #
I have a loop like this
ICollection<Data> GetData()
{
ICollection<Data> result = new List<Data>()
foreach (var item in aCollection)
item.AddData(result);
return result;
}
and now I need this to be done in parallel and not iteratively. My first attempt was to do something like this
ICollection<Data> GetData()
{
ICollection<Data> result = new SynchronizedCollection<Data>()
foreach (var item in aCollection)
new Thread(delegate() { item.AddData(result); }).Start();
return result;
}
but I need to wait for all the data to be added before I return the result.
What's the easiest way to do this?
Edit: this one AddData
will call over the net. The collection usually contains up to several dozen records.
source to share
You can use Parallel Extensions that will be integrated into C # 4.0 and available for C # 3.5 as a separate library that you can download here: Microsoft Parallel Extensions to .NET Framework 3.5 .
If you don't want to use parallel extensions, don't start a thread for every iteration, use ThreadPool instead, which provides better performance.
As I found out in the comments to this answer, List <> is not thread safe, why you should use SynchronizedCollection <> instead.
Here is some sample code using a thread pool.
IEnumerable<object> providers = null;
var waitHandles = new List<WaitHandle>();
foreach (var provider in providers)
{
var resetEvent = new ManualResetEvent(false);
waitHandles.Add(resetEvent);
ThreadPool.QueueUserWorkItem(s =>
{
// do whatever you want.
((EventWaitHandle)s).Set();
}, resetEvent);
}
WaitHandle.WaitAll(waitHandles.ToArray());
source to share
If AddData is really, really slow, the overhead of creating new threads is likely to be a bottleneck in your code.
If you want to pipe this to other threads, you must use ThreadPool to do this. As for waiting for threads to finish, you can use wait commands to signal between threads.
source to share
The easiest way is probably to wait for the parallel framework to arrive in 4.0.
Either way, first you need to be careful to create a thread there. If you have several hundred or even thousands of items, this code could potentially kill your application, or at least your performance.
I would try to design the AddData method as an asynchronous call. IAsyncResult has a WaitHandle. You can wait on these descriptors until all asynchronous messages indicate that the operation is complete. Then you can go on. Running this asynchronous process should ensure that you are using threads from the thread pool. This way, you get pretty good performance, and you achieve natural saturation of workloads if your list gets really big.
source to share
Store references to the stream objects you created and invoke a connection on each one:
ICollection<Data> GetData()
{
ICollection<Data> result = new SynchronizedCollection<Data>()
List<Thread> threads=new List<Thread>();
foreach (var item in aCollection)
{
Thread thread=new Thread(delegate() { item.AddData(result); });
thread.Start();
threads.Add(thread);
}
foreach(Thread thread in threads)
thread.Join();
return result;
}
source to share