Accidental application of segmentation fault in python extension functions in C ++
I am experiencing some old problem in python extension in C ++. I appreciate any help or suggestions.
Set context.
I am using embedded python in C ++ to execute python scripts. I am also using python extension in C ++ to enable the python script to call a C ++ function.
What is the problem?
-
When I comment out the method input from modules "method table" as shown below. The app never crashes, or I don't experience a segfault at all.
static PyMethodDef sa_methods[] = { //{"GetBlue",(PyCFunction)sa_GetBlue,METH_VARARGS,PyDoc_STR("fetches Blue color")}, //{"GetRed",(PyCFunction)sa_GetRed,METH_VARARGS,PyDoc_STR("fetches Red color")}, {"GetYellow",(PyCFunction)sa_GetYellow,METH_VARARGS,PyDoc_STR("fetches yellow color")}, {"GetPink",(PyCFunction)sa_GetPink,METH_VARARGS,PyDoc_STR("fetches pink color")}, {NULL, NULL, 0, NULL} };
-
When I remove the method comments from the method entry from the method table modules like below. The application always crashes randomly between processing 20,000 - 50,000 requests.
static PyMethodDef sa_methods[] = { {"GetBlue",(PyCFunction)sa_GetBlue,METH_VARARGS,PyDoc_STR("fetches Blue color")}, {"GetRed",(PyCFunction)sa_GetRed,METH_VARARGS,PyDoc_STR("fetches Red color")}, {"GetYellow",(PyCFunction)sa_GetYellow,METH_VARARGS,PyDoc_STR("fetches yellow color")}, {"GetPink",(PyCFunction)sa_GetPink,METH_VARARGS,PyDoc_STR("fetches pink color")}, {NULL, NULL, 0, NULL} };
I kept the above two methods empty and they just return Py_False.
Below find the stack trace that I get every time.
#0 0x00002b700146c4d7 in PyNumber_CoerceEx (pv=0x41801798, pw=0x41801790) at Objects/object.c:1599
#1 0x00002b700142d6e3 in binary_op1 (v=0x2aaaac079600, w=0x2b7001750550, op_slot=16) at Objects/abstract.c:929
#2 0x00002b7001431e08 in PyNumber_Multiply (v=0x41801798, w=0x41801790) at Objects/abstract.c:1188
#3 0x00002b70014c4326 in PyEval_EvalFrameEx (f=0x8127780, throwflag=<value optimized out>) at Python/ceval.c:1118
#4 0x00002b70014c8493 in call_function (f=0x996e660, throwflag=<value optimized out>) at Python/ceval.c:3792
#5 PyEval_EvalFrameEx (f=0x996e660, throwflag=<value optimized out>) at Python/ceval.c:2389
#6 0x00002b70014c8493 in call_function (f=0x2aaac30aeb50, throwflag=<value optimized out>) at Python/ceval.c:3792
#7 PyEval_EvalFrameEx (f=0x2aaac30aeb50, throwflag=<value optimized out>) at Python/ceval.c:2389
#8 0x00002b70014c8d9f in PyEval_EvalCodeEx (co=0x2aaaad7dc558, globals=<value optimized out>, locals=<value optimized out>, args=0x2aaac6ab59a0, argcount=2,
kws=0x2aaac6ab59b0, kwcount=0, defs=0x7819128, defcount=1, closure=0x0) at Python/ceval.c:2968
#9 0x00002b70014c6df3 in call_function (f=0x2aaac6ab57d0, throwflag=<value optimized out>) at Python/ceval.c:3802
#10 PyEval_EvalFrameEx (f=0x2aaac6ab57d0, throwflag=<value optimized out>) at Python/ceval.c:2389
#11 0x00002b70014c8d9f in PyEval_EvalCodeEx (co=0x2aaabdf8e828, globals=<value optimized out>, locals=<value optimized out>, args=0x2aaac65bcbf0, argcount=2,
kws=0x2aaac65bcc00, kwcount=0, defs=0x7819c28, defcount=1, closure=0x0) at Python/ceval.c:2968
#12 0x00002b70014c6df3 in call_function (f=0x2aaac65bca40, throwflag=<value optimized out>) at Python/ceval.c:3802
#13 PyEval_EvalFrameEx (f=0x2aaac65bca40, throwflag=<value optimized out>) at Python/ceval.c:2389
#14 0x00002b70014c8493 in call_function (f=0x2aaac6301630, throwflag=<value optimized out>) at Python/ceval.c:3792
#15 PyEval_EvalFrameEx (f=0x2aaac6301630, throwflag=<value optimized out>) at Python/ceval.c:2389
#16 0x00002b70014c8d9f in PyEval_EvalCodeEx (co=0x6c7f558, globals=<value optimized out>, locals=<value optimized out>, args=0x2aaac55963e8, argcount=2,
kws=0x2aaac55963f8, kwcount=0, defs=0xd83fce8, defcount=1, closure=0x0) at Python/ceval.c:2968
#17 0x00002b70014c6df3 in call_function (f=0x2aaac5596240, throwflag=<value optimized out>) at Python/ceval.c:3802
#18 PyEval_EvalFrameEx (f=0x2aaac5596240, throwflag=<value optimized out>) at Python/ceval.c:2389
#19 0x00002b70014c8d9f in PyEval_EvalCodeEx (co=0x6c7fcd8, globals=<value optimized out>, locals=<value optimized out>, args=0x2aaac6fbc328, argcount=2,
kws=0x2aaac6fbc338, kwcount=0, defs=0xd83fc68, defcount=1, closure=0x0) at Python/ceval.c:2968
#20 0x00002b70014c6df3 in call_function (f=0x2aaac6fbc1a0, throwflag=<value optimized out>) at Python/ceval.c:3802
#21 PyEval_EvalFrameEx (f=0x2aaac6fbc1a0, throwflag=<value optimized out>) at Python/ceval.c:2389
#22 0x00002b70014c8d9f in PyEval_EvalCodeEx (co=0x6c7f648, globals=<value optimized out>, locals=<value optimized out>, args=0x1, argcount=0, kws=0x2aaac3e32558,
kwcount=0, defs=0xd83ffa8, defcount=1, closure=0x0) at Python/ceval.c:2968
#23 0x00002b70014c6df3 in call_function (f=0x2aaac3e323d0, throwflag=<value optimized out>) at Python/ceval.c:3802
#24 PyEval_EvalFrameEx (f=0x2aaac3e323d0, throwflag=<value optimized out>) at Python/ceval.c:2389
#25 0x00002b70014c8d9f in PyEval_EvalCodeEx (co=0x2aaaad7f70a8, globals=<value optimized out>, locals=<value optimized out>, args=0x0, argcount=0, kws=0x1325c6a8,
kwcount=0, defs=0x0, defcount=0, closure=0x2aaaad18e990) at Python/ceval.c:2968
#26 0x00002b70014c6df3 in call_function (f=0x1325c480, throwflag=<value optimized out>) at Python/ceval.c:3802
#27 PyEval_EvalFrameEx (f=0x1325c480, throwflag=<value optimized out>) at Python/ceval.c:2389
#28 0x00002b70014c8d9f in PyEval_EvalCodeEx (co=0x2aaaad7f77b0, globals=<value optimized out>, locals=<value optimized out>, args=0x1, argcount=1, kws=0x2aaac5ec1d08,
kwcount=0, defs=0x2aaaacfdf5e8, defcount=1, closure=0x0) at Python/ceval.c:2968
#29 0x00002b70014c6df3 in call_function (f=0x2aaac5ec1b00, throwflag=<value optimized out>) at Python/ceval.c:3802
#30 PyEval_EvalFrameEx (f=0x2aaac5ec1b00, throwflag=<value optimized out>) at Python/ceval.c:2389
#31 0x00002b70014c8493 in call_function (f=0xa926080, throwflag=<value optimized out>) at Python/ceval.c:3792
#32 PyEval_EvalFrameEx (f=0xa926080, throwflag=<value optimized out>) at Python/ceval.c:2389
#33 0x00002b70014c8493 in call_function (f=0x8e4bd40, throwflag=<value optimized out>) at Python/ceval.c:3792
#34 PyEval_EvalFrameEx (f=0x8e4bd40, throwflag=<value optimized out>) at Python/ceval.c:2389
#35 0x00002b70014c8d9f in PyEval_EvalCodeEx (co=0x2aaabc9f7648, globals=<value optimized out>, locals=<value optimized out>, args=0x9ff5f58, argcount=1,
---Type <return> to continue, or q <return> to quit---
kws=0x9ff5f60, kwcount=0, defs=0x0, defcount=0, closure=0x2aaaacfdfc50) at Python/ceval.c:2968
#36 0x00002b70014c6df3 in call_function (f=0x9ff5dc0, throwflag=<value optimized out>) at Python/ceval.c:3802
#37 PyEval_EvalFrameEx (f=0x9ff5dc0, throwflag=<value optimized out>) at Python/ceval.c:2389
#38 0x00002b70014c8493 in call_function (f=0xdb993e0, throwflag=<value optimized out>) at Python/ceval.c:3792
#39 PyEval_EvalFrameEx (f=0xdb993e0, throwflag=<value optimized out>) at Python/ceval.c:2389
#40 0x00002b70014c8d9f in PyEval_EvalCodeEx (co=0x2aaabd820990, globals=<value optimized out>, locals=<value optimized out>, args=0x2aaaacfdf8a8, argcount=1, kws=0x0,
kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2968
#41 0x00002b700145815d in function_call (func=0x7d898c0, arg=0x2aaaacfdf890, kw=0x0) at Objects/funcobject.c:524
#42 0x00002b700142d318 in PyObject_Call (func=0x7d898c0, arg=0x2aaaacfdf890, kw=0x0) at Objects/abstract.c:2492
#43 0x00002b700143cd7f in instancemethod_call (func=<value optimized out>, arg=0x2aaaacfdf890, kw=0x0) at Objects/classobject.c:2579
#44 0x00002b700142d318 in PyObject_Call (func=0x7554690, arg=0x2aaaac04d050, kw=0x0) at Objects/abstract.c:2492
#45 0x00002b70014c0e16 in PyEval_CallObjectWithKeywords (func=0x7554690, arg=0x2aaaac04d050, kw=0x0) at Python/ceval.c:3575
#46 0x00002b70014f9dcd in t_bootstrap (boot_raw=0x2aaac76e67c0) at ./Modules/threadmodule.c:425
#47 0x00000033c720677d in start_thread () from /lib64/libpthread.so.0
#48 0x00000033c66d33ed in clone () from /lib64/libc.so.6
+3
source to share
1 answer
I came up with a solution to this problem, which ultimately gave me and my client a huge amount of relief.
And here's the reason and the solution.
The "sa_GetBlue" and "sa_GetRed" functions returned "Py_False". Before returning "Py_False" or "Py_True" you need to increment the reference count of the variable holding these values, otherwise it may corrupt the interpreter's memory state; it will not crash immediately, but it can crash at any point in time. In my case, it crashes after processing requests from 20,000 to 50,000.
Please check below link to find out more about what I am trying to explain.
why-does-python-keep-a-reference-count-on-false-and-true
@doomster and @Omnifarious; Thank you for your comment and guidance provided.
+4
source to share