Python: is there check_call output to file?
Is it possible to get the following procedure check_call
:
logPath="log.txt"
with open(logPath,"w") as log:
subprocess.check_call(command, stdout = log, stderr=subprocess.STDOUT )
to output stdout
and stderr
to a file continuously ?
On my machine, the output is written to a file only after subprocess.check_call finished
.
To achieve this, perhaps we could change the length of the stream buffer log
?
source to share
Not without some OS tricks.
This is because the output is usually line-buffered (i.e. after a newline, the buffer is flushed ) when the output is terminal, but is blocked by a buffer when the output is a file or pipe, so in block buffering you won't see output written "continuously ", but rather it will be recorded every 1k or 4k or whatever the block size.
This is the default behavior for libc, so if the subprocess is written in C and with printf()
/ fprintf()
, it checks the output if it is a terminal or a file and changes the buffering mode accordingly.
The buffering concept (better) is explained at http://www.gnu.org/software/libc/manual/html_node/Buffering-Concepts.html
This is done for performance reasons (see the answer to this question ).
If you can change the code of the subprocess, you can place the call flush()
after each line or as needed.
Otherwise, there are external tools for enforcing line buffering (by tricking programs that read output as a terminal):
Possibly related:
- Force line-buffering stdout when piping to tee (assumes use
unbuffer
) - java subprocess doesn't write its output until after it finishes (shorter explanation I wrote years ago)
- How do I get the "instant" output of "tail -f" as an input file? (suggests
stdbuf
usage) - Grep pipes not working with tail? (only for
grep
)
source to share