PipedInputStream / PipedOutputStream, ImageIO and ffmpeg

I have the following code in Scala:

      val pos = new PipedOutputStream()
      val pis = new PipedInputStream(pos)

      Future {
        LOG.trace("Start rendering")
        generateFrames(videoRenderParams.length) {
          img ⇒ ImageIO.write(img, "PNG", pos)
        }
        pos.flush()
        IOUtils.closeQuietly(pos)
        LOG.trace("Finished rendering")
      } onComplete {
        case Success(_) ⇒ 
          LOG.trace("Complete successfully")
        case Failure(err) ⇒
          LOG.error("Can't render stuff", err)
          IOUtils.closeQuietly(pis)
          IOUtils.closeQuietly(pos)
      }

      val prc = (ffmpegCli #< pis).!(logger)

      

The future just writes the generated images one by one to the OutputStream . The ffmpeg process now reads the input images from stdin and converts them to an MP4 file.

This works very well, but for some reason I sometimes end up with the following stacks:

I/O error Pipe closed for process: <input stream>
java.io.IOException: Pipe closed
    at java.io.PipedInputStream.checkStateForReceive(PipedInputStream.java:260)
    at java.io.PipedInputStream.receive(PipedInputStream.java:226)
    at java.io.PipedOutputStream.write(PipedOutputStream.java:149)
    at scala.sys.process.BasicIO$.loop$1(BasicIO.scala:236)
    at scala.sys.process.BasicIO$.transferFullyImpl(BasicIO.scala:242)
    at scala.sys.process.BasicIO$.transferFully(BasicIO.scala:223)
    at scala.sys.process.ProcessImpl$PipeThread.runloop(ProcessImpl.scala:159)
    at scala.sys.process.ProcessImpl$PipeSource.run(ProcessImpl.scala:179)

      

At the same time, I am getting the following error from another thread:

javax.imageio.IIOException: I/O error writing PNG file!
    at com.sun.imageio.plugins.png.PNGImageWriter.write(PNGImageWriter.java:1168)
    at javax.imageio.ImageWriter.write(ImageWriter.java:615)
    at javax.imageio.ImageIO.doWrite(ImageIO.java:1612)
    at javax.imageio.ImageIO.write(ImageIO.java:1578)
    at 

      

So, it seems that the streams were split somewhere in the middle, so ffmpeg cannot read data, and ImageIO cannot write data.

What's even more interesting is that the problem is reproducible only on a specific Linux server (Amazon). It works flawlessly on other Linux boxes. So I am wondering if someone can point me to the possible causes of this error.

What I've tried so far:

  • use Oracle JDK 8 and OpenJDK
  • use different versions of FFMPEG

Nothing worked at the moment.

+3


source to share


1 answer


The problem was pretty predictable and strange at the same time. Thus, ten parallel ffmpeg processes were scheduled to process the input , and the input was a set of hundreds of FullHD images . Obviously, this requires a lot of processing power, so the kernel accidentally shuts down ffmpeg processes , forcing the Java shell to report invalid input and output pipes at the same time.

So / var / log / messages contains many logs as shown below:



Out of memory: Kill process 25778 (java) score 159 or sacrifice child
Killed process 25931 (ffmpeg) total-vm:2337040kB, anon-rss:966340kB, file-rss:104kB

      

Reducing the number of parallel ffmpeg processes solved the problem.

+3


source







All Articles