Thread pools and context switching (tasks)?
This is a fairly general question in computer science and is not specific to any OS or framework.
As such, I'm a little confused about the overhead associated with switching tasks in the thread pool. In many cases, it doesn't make sense to give each job its own specific thread (we don't want to create too many hardware threads), so instead we put those jobs into tasks that can be scheduled to run on a thread. We set up a thread pool and then dynamically allocate tasks to run on a thread taken from the thread pool.
I'm a little confused (can't find a detailed answer) about the overhead associated with switching tasks on a specific thread (thread pool). DrDobbs's article (see below) states that it works, but I need a deeper answer to what is actually going on (a source able to quote would be fantastic :)).
By definition, SomeWork should be queued in the pool and then start a different thread than the original thread. This means that we will definitely incur the overhead in the queue and context switch to move the work to the pool. If we need to communicate a response to the original thread, for example through a message or Future or the like, we carry another context switch for that.
Source: http://www.drdobbs.com/parallel/use-thread-pools-correctly-keep-tasks-sh/216500409?pgno=1
What components of the stream actually switch? The stream itself does not actually switch, only the stream specific data. What are the overheads associated with this (more, less, or the same)?
source to share
Let's first clarify five basic concepts and then discuss how they relate in the context of a thread pool:
-
Subject: In a short summary, it can be described as the context of program execution, given by the executable code, data in processor registers, and the stack. when a thread is created, it is assigned code to run in that thread context. In each processor cycle, a thread has an instruction to execute and data in the processor registers and a stack in a given state.
-
Task: Represents a unit of work. This is the code assigned to the thread that needs to be executed.
-
context switch (from wikipedia): Is the process of storing and restoring the state (context) of a thread so that execution can be resumed from the same point later. This allows multiple processes to share a single processor and is an important feature of a multitasking operating system. What makes up the context, as described above, is the executable code, processor registers, and the stack.
What context switches is flow. A task is only a world of work that can be assigned to a thread to be executed. At the moment, the thread can execute the task.
-
Thread pool (from wikipedia): In computer programming, a thread pool is where multiple threads are created to perform a series of tasks, which are usually queued up.
-
Thread pool queue: Where tasks are placed to be executed by threads in the pool. This data structure is a shared memory world where threads can compete with queue / deactivation, can lead to conflict in high load scenarios.
Illustrating a thread pool usage scenario:
- In your program (ultimately running on the main thread), you create a task and schedule it to run in the thread pool.
- The task is queued in the thread pool queue.
- When a pooled thread is running, it removes the task from the pool and starts executing it.
- If there is no free processor to execute a thread from the pool, the operating system at some point (depending on the thread scheduler policy and thread priorities) will stop the thread from executing, switching the context to another thread.
the operating system can stop the execution of a thread at any time by switching the context to another thread, returning it to continue where it left off.
The context switch overhead adds up as the number of active threads competing for cpus grows. Thus, ideally, the thread pool tries to use the minimum required threads to occupy all the available processors on the machine.
Unless your tasks have code that blocks somewhere, context switching is minimized as it uses more threads than the available CPU on the machine.
Of course, if you only have one core, your main thread and thread pool will be competing for the same processor.
source to share
The article is probably talking about the case when work is sent to the pool and the result is waiting for it. Running a task in a thread pool generally does not carry any context redistributions.
Imagine there are 1000 work items in the queue. The thread-pool thread will execute them one by one. All without one context switch between them.
The switchover is pending / blocking.
source to share