Problem calling bash from within python
I am trying to use python to help do some automation around an incremental build function in an android build system. Typically, from a given directory, I would run the following command to create what is in that directory and subdirectories:
mm -j8
This is similar to the "make" command, only incremental build and is defined as a function in a bash file named envsetup.sh. That it doesn't matter, just know that it is a function defined in a bash script somewhere on the filesystem. To accomplish this, I can also:
bash -c ". /path/to/envsetup.sh; mm -j8"
This calling method will be essential when calling a function from python. I followed the solution here showing how to call a function in a bash script from python. I used this method in a simple script that in theory should just spit out STDOUT and STDERR from command execution:
import subprocess
command = ['bash', '-c', '. /path/to/envsetup.sh; mm -j8']
(stdout, stderr) = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True).communicate()
print 'stdout: ' + stdout
print 'stderr: ' + stderr
However, the call Popen
never returns. What am I doing wrong that would allow bash to execute the command correctly, but Python hangs when executing the command?
source to share
TL; DR:
Your problem is using shell=True
. Install it on shell=False
and it will work.
With this option, python will just run the first element of the array command
i.e. bash
as a shell script. So, python is currently starting its own shell to run your command ( bash
). It will run bash with no arguments, and bash will wait for input, blocking your python script.
The parameter shell=True
is for cases where you are passing a shell script as a single line. When you explicitly specify the shell and its parameters as the process to invoke, as you do above, you must set shell=False
.
>>> import subprocess
>>> subprocess.Popen(['bash', 'whatever'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True).communicate()
This is what the process tree looks like when I run the above:
\_ python
\_ /bin/sh -c bash whatever
\_ bash
Actually passed whatever
, but it is a parameter sh
, not a parameter to the internal bash
, so the command being executed is efficient ['/bin/sh', '-c', 'bash', 'whatever']
, which is very different from ['/bin/sh', '-c', 'bash whatever']
>
source to share