Using System.cmd within a Poolboy worker (gen_server) causes a silent crash
I have a function that is generated from a machine gun worker
basic overview:
- Phoenix Controller calls dispatcher with data
- The dispatcher transfers data to the Poolboy worker
- Poolboy worker launches new process with data to process
- The new process uses the data to invoke the system command (wget in this case)
The problem I am facing is when I run the ExUnit test, it goes completely to the spawned process and I can output the data (using IO.inspect).
When I run System.cmd ("wget" .... I see wget output in the terminal when I run the ExUnit test, so the command is executed, but then everything I do after that command is t.
So, from my worker, if I do this:
IO.puts "hello"
System.cmd("wget", opts)
IO.puts "world"
Then I see hello
I can see the output from wget but I cannot seeworld
If I do something like:
IO.puts "hello"
File.write("/tmp/temp.txt", "test")
IO.puts "world"
Then I see both hello
and world
and the file is written.
Is there something specific about System.cmd
that I'm missing that is causing this? It works great when not running as a separate process, so it is a combination of process and System.cmd.
Any ideas? Thank!
source to share
You have entered part of the marked Elixir
"Dragons will be here"
System.cmd
is just a simple wrapper around Port
and Port
a largely undocumented wrapper for an Erlang function Port
.
http://www.erlang.org/doc/man/erlang.html#open_port-2
Erlang BEAM's main process scheduler is built on the assumption that it can "swap" processes at very short time intervals. If you only use Erlang / Exilir code, they are all built to run in the BEAM VM. Any code that might block or hang on a system call must run in the driver. It is a special interface in the Erlang VM that isolates the Erlang scheduler from any processes that might hang on system calls.
The port driver is configured to handle calls to external programs.
System.cmd
ultimately causes
do_cmd Port.open({:spawn_executable, cmd}, opts), initial, fun
The port runs in a separate process, and the procedure do_cmd
starts fetching until it receives an exit status from the base Erlang port. So System.cmd will "block" this particular BEAM process until wget unix process exits.
However, the rest of the BEAM Elixir processes will go on their merry road. I'm not familiar enough with PoolBoy to know if there is some kind of monitor timeout or heartbeat on your workers. However, if the wget command is present and exceeds this timeout, the worker process may exit before the wget command exits.
System.cmd
not really tuned in to solve all the problems around the team, which could potentially take a long time. I would suggest you look into the module Porcelain
as a nice wrapper around the rather complex Erlang ports topic.
https://github.com/alco/porcelain
Or, since you are doing a simple wget, using an Elixir or Erlang HTTP client module will probably work much better in a BEAM framework.
source to share