Windows stack protection page does not start in _chkstk
I've seen several crashes "in the wild" where the crash dump shows code throwing an access violation inside _chkstk when trying to expand the stack. Windbg reveals that _chkstk touches the protect page, but instead of expanding the stack as it should, it just throws an access violation.
I suspected it might have something to do with the user-mode exception handlers in the code, however my testing shows that under normal circumstances _chkstk watchdog exceptions happen in kernel mode and don't even reach the user-mode exception handlers.
Hence, in this case, it looks like exceptions for kernel mode protection are not handled for whatever reason, and user mode access violations are triggered instead.
What can happen?
source to share
This turned out to be a problem with the XP / Server 2003 kernel. On this OS, if one thread reads another thread stack, then the guard page and TIB state are messed up, and any subsequent attempts to increase the stack (_chkstk) result in an access violation. This doesn't happen on later OSs.
In our case, we were writing an in-process mini-drive containing stream stacks, which would mangle the state of the stack, as described when the dbghelp library read each thread stack.
The moral of this story is that it is inefficient to generate in-process mini-pumps, they must always be generated by an external process.
source to share