Java subprocess does not write its output until it completes
I wrote this simple test file that describes the problem I am facing:
I am creating a subprocess from Java and simultaneously starting a thread that MUST write each line as soon as it reads from the standard output of the subprocess.
Instead, I get that the output of the subprocess is completely written when it exits. Here's the result:
Mon Jul 15 19:17:13 CEST 2013: starting process Mon Jul 15 19:17:14 CEST 2013: process started Mon Jul 15 19:17:14 CEST 2013: waiting for process termination Mon Jul 15 19:17:14 CEST 2013: readerThread is starting Mon Jul 15 19:17:19 CEST 2013: process terminated correctly Mon Jul 15 19:17:19 CEST 2013: Thread[Thread-0,5,main] got line: foo(7) Mon Jul 15 19:17:19 CEST 2013: Thread[Thread-0,5,main] got line: foo(49) Mon Jul 15 19:17:19 CEST 2013: Thread[Thread-0,5,main] got line: foo(73) Mon Jul 15 19:17:19 CEST 2013: Thread[Thread-0,5,main] got line: foo(58) Mon Jul 15 19:17:19 CEST 2013: Thread[Thread-0,5,main] got line: foo(30) Mon Jul 15 19:17:19 CEST 2013: readerThread is terminating
with this code:
public class MiniTest {
static void println(String x) {
System.out.println(new Date() + ": " + x);
}
public static void main(String[] args) throws IOException, InterruptedException {
ProcessBuilder pb = new ProcessBuilder("bin/dummy", "foo", "5");
println("starting process");
Process p = pb.start();
println("process started");
new ReaderThread(p).start();
println("waiting for process termination");
p.waitFor();
println("process terminated correctly");
}
static class ReaderThread extends Thread {
private Process p;
public ReaderThread(Process p) {
this.p = p;
}
public void run() {
println("readerThread is starting");
BufferedReader r = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line;
try {
while((line = r.readLine()) != null) {
println(this + " got line: " + line);
}
} catch(IOException e) {
println("read error: " + e);
}
println("readerThread is terminating");
}
}
}
Note: the subprocess is very simple, it prints out a line every second, for a certain number of iterations (and when testing on the command line, it does this):
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char **argv) {
char *f = argv[1];
int n = atoi(argv[2]);
while(n-- > 0) {
printf("%s(%d)\n", f, rand() % 100);
sleep(1);
}
return 0;
}
source to share
Found the answer by asking and studying the question.
The problem cannot be easily solved, but it appears to be a libc issue. If the output is not tty / pty, the buffering is not line-by-line.
There are external programs or script, however this allows you to bypass this, for example stdbuf or unbuffer (from expect).
Detailed explanation: Forcing line-buffering stdout when piping to tee
source to share