AnyEvent :: Fork how to wait for a baby
I am considering a module AnyEvent::Fork
. I have 20 external scripts that I would like to call in parallel (6 at a time) and summarize their output later when it's over. I am at a loss on how to achieve this.
The sample code (which only calls 1 child) in the module has a problem. I just add a simple sleep to the code so that it doesn't come back right away and the parent exits immediately without waiting for the child process.
open my $output, ">/tmp/log" or die "$!";
AnyEvent::Fork
->new
->eval ('
# compile a helper function for later use
sub run {
my ($fh, $output, @cmd) = @_;
# perl will clear close-on-exec on STDOUT/STDERR
open STDOUT, ">&", $output or die;
open STDERR, ">&", $fh or die;
### Added by me to demonstrate that
### $cv->recv returns immediately.
sleep 5;
exec @cmd;
}
')
->send_fh ($output)
->send_arg ("/bin/echo", "hi")
->run ("run", my $cv = AE::cv);
my $stderr = $cv->recv;
The result is /tmp/log
empty. I don't understand how it is condvar
used here, it is not in the documentation. Can I get the number of working children with help condvar
?
Please help how to do this correctly.
UPDATE the main issue is that the parent was not waiting for the child to complete.
source to share
The problem here is that the parent and child processes are separate processes and can run independently of each other, so explicit synchronization is required if you have to wait. There are many ways to do this, the simplest would be to use AnyEvent :: Fork :: RPC and send, say, a "wait" request for the child, which he replies to when he's finished.
To do this with bare AnyEvent :: Fork, the easiest way is to use the bidirectional pipe provided by -> run:
AnyEvent::Fork
->new
->run (sub {
my ($fh) = @_;
sysread $fh, my $dummy, 1; # will wait for data, or eof
... done, you can now call e.g. $cv->send
sysread will try to read from the child. If the child never sends anything, this will block the parent until the child exits, since at that point the child will close its end of the pipe and sysread will receive an EOF.
Of course, in your AnyEvent program, you probably don't want to block, so you use an I / O watcher:
->run (sub {
my ($fh) = @_;
my $rw; $rw = AE::io $fh, 0, sub {
... read data received, or EOF
undef $rw;
... done, you can now call e.g. $cv->send;
}
});
This trick can be used for external commands as well (exec in your example) by clearing the close state on the pipe in the child and thus passing it to the exec'ed program - in this case, when all programs that inherit the pipe's output are pipe will signal EOF.
This should get you started. There are other ways to do this, but most or even all good ones will include some communication channels, and the easiest is to get the one provided by AnyEvent :: Fork.
source to share