Using Parallel.For to validate SQL queries and compare against ThreadPool
I'm looking for a way to easily load a test and compare some of our SQL (using ADO.NET, nothing fancy using LINQ or PLINQ) that needs to be done when running under high concurrent load.
I thought about using the new concurrent CTP extensions, and in particular Parallel.For
/ Parallel.ForEach
, to just run SQL for over 10,000 iterations or so - but I couldn't find any data on what they were optimized for.
Basically, what I'm concerned about is that since database access is inherently I / O related, it doesn't put enough load. Does anyone know if Parallel. Is it reasonable enough for this to use> x threads (where x = # of processors) if the tasks being performed are not completely CPU bound? That is, it behaves similarly to a managed thread pool?
It would be great if it were!
EDIT: As @CVertex kindly links below, you can set the number of threads independently. Does anyone know if the default parallel libraries are smart enough to keep adding threads if the job is I / O related?
source to share
Of course it can!
You can specify the maximum number of threads for each desired processor.
Before you Parallel.For what you are doing, you will have to instantiate your own TaskManager from the System.Threading.Tasks namespace. Take a look at the ctor options to see how you can customize the task manager for your purposes.
There must be an overload for Parallel. This requires an instance of Task Manager.
source to share
Another way is to define the PLINQ_DOP environment variable. From the docs:
PLINQ_DOP
DOP stands for degree of parallelism. Setting this environment variable defines the number of threads for PLINQ to use.
E.g. PLINQ_DOP=1 means single-threaded, while PLINQ_DOP=8 means PLINQ should use 8 threads.
If this is set to a value greater than the number of procs*cores available on the system,
PLINQ will use more threads than processors. If one of them blocks, for instance,
this allows other threads to make forward progress.
source to share