Advantages of subprocess over os.system
I recently came across several Stack Overflow posts stating that subprocess is much better than os.system, however I am having a hard time finding the exact benefits.
Some examples of things I've come across: https://docs.python.org/3/library/os.html#os.system
"The subprocess module provides more powerful tools for creating new processes and obtaining their results, using this module is preferred over using this function."
I don't know how it is more powerful, but I know that in many ways it is easier to use a subprocess, but how much more efficient is it actually?
Another example:
The advantage of subprocess vs system is that it is more flexible (you can get stdout, stderr, "real" status code, better error handling, etc ...).
This post that has 2600+ votes. Again, couldn't find any information on what was meant by improved error handling or actual status code.
Best comment on this post:
Can't figure out why you are using os.system even for quick / dirty / one-off use. subprocess seems much better.
Again, I understand that some things are a little easier, but I can hardly understand why, for example:
subprocess.call("netsh interface set interface \"Wi-Fi\" enable", shell=True)
better than
os.system("netsh interface set interface \"Wi-Fi\" enabled")
Can anyone explain some reasons this is much better?
source to share
First of all, you cut out the middleman; subprocess.call
by default avoids spawning the shell that checks your command and spawns the requested process directly. This is important because, aside from the efficiency side, you don't have much control over the default shell behavior, and it usually works against you in terms of escaping.
In particular, you usually never do this:
subprocess.call("netsh interface set interface \"Wi-Fi\" enable")
So
If you pass one line, either it
shell
should beTrue
(see below), or the line should just indicate the program to be executed without specifying any arguments.
Instead, you would do:
subprocess.call(["netsh", "interface", "set", "interface", "Wi-Fi", "enable"])
Notice that here all the passing nightmares are gone. subprocess
handles escaping (if the OS requires the arguments as a single string - for example Windows) or passes the split arguments directly to the appropriate syscall ( execvp
on UNIX).
Compare that to having to deal with it, especially in a cross-platform way ( cmd
doesn't disappear in the same way as POSIX sh
), especially with a shell in the middle of messing with your stuff (trust me, you don't want to know how wicked mess - provide 100% safe escaping for your command when invoked cmd /k
).
Also, when used subprocess
without a shell in the middle, you are sure you are getting the correct return codes. If the process fails when starting, you will get a Python exception, if you get a return code, this is actually the return code of the running program. Since os.system
you have no way of knowing if the return code comes from the command being run (which is usually the default behavior if the shell manages to run it), or if it is some error from the shell (if it fails to run it).
Besides the split / escaping and return code arguments, you have better control over the running process. Even with subprocess.call
(which is the most basic function of a function by function subprocess
) you can redirect stdin
, stdout
and stderr
possibly link to a running process. check_call
is similar, and it avoids the risk of ignoring the crash code. check_output
covers a general use case for check_call
+, capturing all program output in a string variable.
Once you pass by call
and friends (who are blocked as os.system
), there are even more powerful features - in particular, the object Popen
allows you to work with a running process asynchronously. You can start it, perhaps talk to it through redirected streams, check if it is running from time to time, doing other things, waiting for it to complete, sending signals to it and killing them - anything but a simple synchronous "startup process with standard stdin / stdout / stderr through the shell and wait for it to finish "that os.system
provides.
So, to sum it up, with subprocess
:
- even at the most basic level (
call
and friends), you:- avoid escaping problems by passing a Python argument list;
- avoid using the command line command line;
- you have an exception or a true exit code of the process you started; no confusion with the program / shell exit code;
- have the ability to write to stdout and generally redirect standard streams;
- when you use
Popen
:- you are not limited to a synchronous interface, but you can actually do other things while the subprocess is running;
- you can control the subprocess (check if it is running, talk to it, kill it).
Given that it subprocess
does more than it os.system
can do - and in a safer, more flexible (if you need it) way - there is simply no reason to use system
.
source to share
There are many reasons, but the main reason is mentioned directly in the docstring:
>>> os.system.__doc__
'Execute the command in a subshell.'
For almost all cases where you need a subprocess, it is not desirable to spawn a subshell . This is unnecessary and useless, it adds an extra layer of complexity and introduces several new vulnerabilities and failure modes. Using the module subprocess
cuts out the middleman.
source to share