Prevent debugging session after pause after every bottom exit
I am debugging a process tree using gdb
very handy support with a few least features :
(gdb) set detach-on-fork off
(gdb) set schedule-multiple on
(gdb) set follow-fork-mode parent
(gdb) break PostgresMain
(gdb) break PostmasterMain
and now everything needs to work until I hit one of the future breakpoints at some that have not yet been spawned below.
However, gdb
it seems "useful" to suspend whenever a normal exit fails, or at least blocks the cleanup of the underlying one to bring its parent back wait()
:
(gdb) c
[New process 16505]
process 16505 is executing new program: /home/craig/pg/bdr/bin/pg_config
Reading symbols from /home/craig/pg/bdr/bin/pg_config...done.
[Inferior 2 (process 16505) exited normally]
(gdb) info inferior
Num Description Executable
* 2 <null> /home/craig/pg/bdr/bin/pg_config
1 process 16501 /usr/bin/make
(gdb) inferior 1
[Switching to inferior 1 [process 16501] (/usr/bin/make)]
[Switching to thread 1 (process 16501)]
#0 0x0000003bc68bc502 in __libc_wait (stat_loc=0x7fffffffbc78) at ../sysdeps/unix/sysv/linux/wait.c:30
30 return INLINE_SYSCALL (wait4, 4, WAIT_ANY, stat_loc, 0,
(gdb)
so I have to endlessly:
(gdb) inferior 1 (gdb) c
to continue. About 70 times before I remove the desired breakpoint from the child child child.
I think what happens when gdb
treats the exit of a process as a stop event, and since it is non-stop
set to off
(default), it stops all threads in all slaves when one thread stops. However, this bottom one is finished, this is not a normal stop event, so you cannot just cont
it, you need to switch to another process first.
Is there a way to stop gdb stopping at every bottom exit? I would have expected follow-fork-mode parent
using schedule-multiple on
to do the trick, but gdb
still seems to want to stop when the lower outputs.
I think I am looking for something like "skip proc-exit" or a virtual signal that I can change for the handler policy so that it doesn't stop.
set non-stop on
it seems like it should be the correct answer, but I suspect it's broken for a few subordinates.
If I use non-stop on
then after the first exit trap the internal state gdb
indicates that the bottom 1 is working:
(gdb) info inferior
Num Description Executable
* 1 process 20540 /usr/bin/make
(gdb) info thread
Id Target Id Frame
* 1 process 20540 "make" (running)
(gdb) cont
Continuing.
Cannot execute this command while the selected thread is running.
but the kernel sees it locked on ptrace_stop
:
$ ps -o "cmd,wchan" -p 20540
CMD WCHAN
/usr/bin/make check ptrace_stop
... and it doesn't advance until gdb
it is disconnected or killed. The process signals are ignored, and interrupt
in gdb
no effect.
I am using GNU gdb (GDB) Fedora 7.7.1-18.fc20
on x86_64.
source to share
After stumbling across a post that references it in passing . I found the missing magic set target-async on
next to set non-stop on
.
non-stop mode, as expected, means that gdb won't stop everything when the lower output yields. target-async
required for gdb 7.7 to work correctly; this is the default 7.8.
So the full spell is:
set detach-on-fork off
set schedule-multiple on
set follow-fork-mode parent
set non-stop on
set target-async on
For 7.8, remove target-async on
and add to reduce noise set print symbol-loading off
.
source to share
The next Python extension for gdb will switch to the first bottom one and resume execution after each stop.
This looks like a general hack, but it works. When a process exits it, a flag is set to indicate that it is stopped on exit and then switches back to the original process. gdb
will stop execution by executing a stop event. We check if the stop was caused by our stop event, and if so, we will continue immediately.
The code also sets the breakpoints I'm using and the multiprocessing parameters, so I can just source thescript.py
and run
.
gdb.execute("set python print-stack full")
gdb.execute("set detach-on-fork off")
gdb.execute("set schedule-multiple on")
gdb.execute("set follow-fork-mode parent")
gdb.execute("set breakpoint pending on")
gdb.execute("break PostgresMain")
gdb.execute("break PostmasterMain")
gdb.execute("set breakpoint pending off")
def do_continue():
gdb.execute("continue")
def exit_handler(event):
global my_stop_request
has_threads = [ inferior.num for inferior in gdb.inferiors() if inferior.threads() ]
if has_threads:
has_threads.sort()
gdb.execute("inferior %d" % has_threads[0])
my_stop_request = True
gdb.events.exited.connect(exit_handler)
def stop_handler(event):
global my_stop_request
if isinstance(event, gdb.SignalEvent):
pass
elif isinstance(event, gdb.BreakpointEvent):
pass
elif my_stop_request:
my_stop_request = False
gdb.post_event(do_continue)
gdb.events.stop.connect(stop_handler)
There should be an easier way than this. It's ugly.
source to share