IOCP in a custom thread pool

IOCP = "I / O Completion Port". It is a kernel object built into the Windows OS that gives you a smart way to manage multithreaded asynchronous IO.

In very simplified (and slightly more simplistic) terms, you tell the IOCP about the IO job you want to do. It will execute them asynchronously and maintain a queue of the results of each of these jobs. Your call to inform the IOCP of the job returns immediately (it doesn't block while the IO is in progress). You are returned an object that is conceptually similar to the .NET IAsyncResult ... it allows you to block if you choose, or you can provide a callback, or you can poll periodically to see if the job has completed.

The IOCP uses a thread pool to perform these jobs. The thread pool tries to limit the number of threads to or less processors (this is configurable, but the goal and default is to limit the number of processors). However, the IOCP makes a provision that sometimes tasks on these threads can block. Asycn IO tasks won't block, but you could give it another task. So you can assign another number to the IOCP ... this is the number of threads above the "normal maximum" that the IOCP is allowed to raise because one of the other threads is blocked. The goal is to have up to the "normal maximum" threads actually doing something (ie not blocking). If this happens, then for a while the IOCP will use more threads than the normal max,but it will refuse to create any new threads until it can return to "normal maximum".

This is a short summary, conceptual only, and, as I said, oversimplified. But that should give you a general idea. Jeffrey Richter's books on Windows describe this in detail (but those books are out of print now). You can find these books and they are actually more expensive than they used to be. I think "Advanced Windows" is the title you want, but there may have been an updated version of the book with a different title.