Executing and monitoring multiple instances of an external program in Python
The main program is as follows:
PREPARE PARAMETERS FOR CHILD PROCESSES
subprocess.Popen('python child.py param=example1'.split(' '))
subprocess.Popen('python child.py param=example2'.split(' '))
...
How to make a main program to monitor all instances of a child process that it started and restart it with appropriate parameters if it doesn't work.
The goal of keeping multiple instances of the child process instead of implementing a multi-threaded architect in the main process is to use as many CPUs and databases as possible.
source to share
Keep dict from .pid
child processes as keys and command lines to restart them as appropriate values. i.e:.
childid = []
for cmdline in cmdlines:
p = subprocess.Popen(cmdline.split())
childid[p.pid] = cmdline
os.wait will return whenever any child process ends: it gives you the (pid, exitstatus) child. So just restart and maintain childid
. i.e:.
while mustcontinue:
pid, exitstat = os.wait()
cmdline = childid.pop(pid)
p = subprocess.Popen(cmdline.split())
childid[p.pid] = cmdline
Presumably you have some criteria for when this endless loop ends, I just used mustcontinue
as a name for those criteria here :-).
source to share