.net Parallel and sequential loop
We would like to create a pattern around specific loops in our solution, which will allow them to run sequentially or in parallel, depending on factors. Below is a general view.
Since parallel collections do not have a common interface with regular collections, we need some kind of adapter to write common code.
Specifically around using a delegate addFunc
in the body of a loop, is there anything that could lead to problems in the long run that we might overlook? It works fine, but ...?
Action<SomeType> addFunc;
if(runInParallel)
{
addFunc = concurrentBag.Add;
loopDelegate = Parallel.ForEach;
}
else
{
addFunc = iList.Add;
loopDelegate = Serial.ForEach; // wrapper delegate for foreach
}
loopDelegate(source, item =>
{
SomeType result = longRunningTask(item);
...
addFunc(result); // will this
});
source to share
Curious why not use TPL in .NET 4.0? http://msdn.microsoft.com/en-us/library/dd537609.aspx
There is excellent technical documentation for the considerations they took when developing the TPL, if you can't use .NET 4 you should look at the article and consider some of the questions received there.
Updated based on comment pointing out the obvious.
I would use some kind of syntactic sugar like
ForEach<Tsource>(Predicate<IEnumerable<TSource>> isParallel, IEnumerable<TSource> source, Action<TSource> body)
{
if(isParallel(source))
{
Parallel.ForEach<TSource>(source, body);
}
else
{
foreach (TSource element in source)
{
body(element);
}
}
}
There are two main advantages to your implementation.
- You enumerate twice, once to add items to the loop, and a second time at runtime.
- It's not immediately obvious, but you stop Parallel.ForEach and can't use the most efficient getter. Parallel.ForeEach will not always use GetEnumerator on IEnumerable. An enumerator is not parallel by design, if your element implements IList, Parallel.ForEach will use the index so that every thread can access the element in the source without having to wait for the enumerator to iterate over that list. ConcurrentBag does not implement IList.
This is the document I had in mind as btw, http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=19222 .
Everything reads very well if you mimic this, but pay special attention to Pages 1 [5-7], 26, 3 [0-2].
Syntactic sugar means you can call it the same as with TPL,
MyParllelLibrary.ForEach( (list) => true, list), item =>
{
// What my code does
});
source to share