Preventing Unauthorized Crash in Application Call Library via P / Invoke
I have a C # application App A
that calls through a P / Invoke C ++ (Qt) DLL App B
. Suppose I can't edit in any way App B
, but I know what is App B
throwing Out of Memory Exception
into a set of conditions that I can replicate but not accurately test before passing the input from App A
to App B
. This exception poses no threat to App A
and can be ignored if it App B
can be "reset" in any way, but when a App B
failure causes it, it calls abort()
, which in turn causes App A
the runtime to terminate.
How can I prevent the App B
inevitable, unpredictable and usually earthly collapse from exposure App A
?
Notes:
- This error ignores the implemented one
UnhandledExceptionHandler
. -
App B
the crash is due to a bug in QtWebkit that could have been handledApp B
by deleting the oversized object and returning null. -
App A
does not report out of memory and there is more than enough memory on the computer to perform the operationApp B
multiple times regardless of the error, but this memory does not appear to be allocatedApp B
for any reason.
source to share
You can handle SIGABRT from unmanaged code, but this is a little messy. It might be possible to do this entirely in C #, but my method requires a C ++ / CLI build in order for it to work (at least it's all in one process).
Step 1. Create a C ++ / CLI class library project, in * .h file put the following function
namespace ClassLibrary1 {
public ref class Class1
{
public:
static void callaborter();
};
}
Step 2: in the cpp file enter the following code:
// This is the main DLL file.
#include "stdafx.h"
#include <csetjmp>
#include <csignal>
#include <cstdlib>
#include <iostream>
#include "ClassLibrary1.h"
#include "..\Win32Project1\Win32Project1.h"
#pragma managed(push, off)
jmp_buf env;
void on_sigabrt (int signum)
{
longjmp (env, 1);
}
void run()
{
if (setjmp (env) == 0) {
signal(SIGABRT, &on_sigabrt);
fnWin32Project1();
}
else {
std::cout << "aborted\n";
}
}
#pragma managed(pop)
void ClassLibrary1::Class1::callaborter()
{
run();
}
I left my class library and class names by default, and my interrupt function was a call to fnWinProject1 (). You will need to link the malicious DLL directly to this project (as in C ++)
In your C # class, include the C ++ / CLI assembly reference and call the "callaborer" method.
The running code will set a hook for SIGABRT (called by the abort () function) and processed in on_sigabrt (), which will cause the entire function to close. Then you can clean up and try again.
I'll leave this as an exercise for the OP to pick out useful names.
Note. In Debug you will still get the undo dialog, just click Ignore to continue. In Release, you won't see the dialog.
PS I must mention that I built my answer based on this stackoverflow question: How to handle the SIGABRT signal?
source to share