Create and include a file-like object as input for the command
I'm looking for a better way to do this, if possible:
import subprocess
f = open('temp.file', 'w+')
f.write('hello world')
f.close()
out = subprocess.check_output(['cat', 'temp.file'])
print out
subprocess.check_output(['rm', 'temp.file'])
In this example, I am creating a file and passing it as input to cat
(actually it is not cat
, but I run, but another program that parses the input file pcap
).
I am wondering if there is a way in Python I can create a "file-like object" with some content and pass this file-like object as input to the command line program. If possible, I believe it would be more efficient than writing the file to disk and then deleting that file.
source to share
If the program is configured to read from stdin
, you can use Popen.communicate :
>>> from subprocess import Popen, PIPE
>>> p = Popen('cat', stdout=PIPE, stdin=PIPE, stderr=PIPE)
>>> out, err = p.communicate(input=b"Hello world!")
>>> out
'Hello world!'
source to share
check_output
takes an input argument stdin
to specify a file-like object to connect to the process's standard inputs.
with open('temp.file') as input:
out = subprocess.check_output(['cat'], stdin=input)
Also, there is no need to run the command rm
; you can delete the file directly from Python:
os.remove('temp.file')
source to share
You can write to TemporaryFile
import subprocess
from tempfile import TemporaryFile
f = TemporaryFile("w")
f.write("foo")
f.seek(0)
out = subprocess.check_output(['cat'],stdin=f)
print(out)
b'foo'
If you just want to write to file as object and get content:
from io import StringIO
f = StringIO()
f.write("foo")
print(f.getvalue())
source to share
If the command only accepts filenames, if it doesn't read data from its stdin, i.e. if you can't use + or , you can try filenames: stdin=PIPE
.communicate()
stdin=real_file
/dev/fd/#
#!/usr/bin/env python3
import os
import subprocess
import threading
def pump_input(pipe):
with pipe:
for i in range(3):
print(i, file=pipe)
r, w = os.pipe()
try:
threading.Thread(target=pump_input, args=[open(w, 'w')]).start()
out = subprocess.check_output(['cat', '/dev/fd/'+str(r)], pass_fds=[r])
finally:
os.close(r)
print('got:', out)
Content does not touch the disc. The input is passed to the sub-process directly through the channel.
If you have a file-like object that is not a real file (otherwise just pass its name as a command line argument) then it pump_input()
might look like this:
import shutil
def pump_input(pipe):
with pipe:
shutil.copyfileobj(file_like_object, pipe)
source to share