How to set presentation timestamp in Android MediaCodec?
I am doing h.264 video rendering on Android using MediaCodec (and SurfaceView). Here are some of my codes.
@Override
public void surfaceCreated(SurfaceHolder holder) {
}
@Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
if (videoPlayer == null) {
videoPlayer = new PlayerThread(holder.getSurface());
videoPlayer.start();
}
}
@Override
public void surfaceDestroyed(SurfaceHolder holder) {
if (videoPlayer != null) {
videoPlayer.interrupt();
videoPlayer.isEOS = true;
videoPlayer = null;
}
}
private class PlayerThread extends Thread {
private MediaCodec decoder;
private Surface surface;
private final static String mimeType = "video/avc";
public boolean isEOS = false;
public PlayerThread(Surface surface) {
this.surface = surface;
}
@Override
public void run() {
MediaFormat format = MediaFormat.createVideoFormat(mimeType, frame.width, frame.height);
decoder = MediaCodec.createDecoderByType(mimeType);
decoder.configure(format, surface, null, 0);
decoder.start();
ByteBuffer[] inputBuffers = decoder.getInputBuffers();
ByteBuffer[] outputBuffers = decoder.getOutputBuffers();
int startPTS = 0;
while (!Thread.interrupted() && !isEOS) {
frame = frameReader.nextFrame();
if (startPTS == 0) {
startPTS = frame.pts;
}
int relativePTS = frame.pts - startPTS;
int inIndex = decoder.dequeueInputBuffer(-1);
if (inIndex >= 0)
{
ByteBuffer inputBuffer = inputBuffers[inIndex];
inputBuffer.clear();
inputBuffer.put(frame.buf, 0, frame.size);
decoder.queueInputBuffer(inIndex, 0, frame.size, relativePTS*1000, 0);
}
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
int outIndex = decoder.dequeueOutputBuffer(info, -1);
switch (outIndex) {
case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:
Log.d("DecodeActivity", "INFO_OUTPUT_BUFFERS_CHANGED");
outputBuffers = decoder.getOutputBuffers();
break;
case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
Log.d("DecodeActivity", "New format " + decoder.getOutputFormat());
break;
case MediaCodec.INFO_TRY_AGAIN_LATER:
Log.d("DecodeActivity", "dequeueOutputBuffer timed out!");
break;
default:
ByteBuffer outputBuffer = outputBuffers[outIndex];
Log.v("DecodeActivity", "We can't use this buffer but render it due to the API limit, " + outputBuffer);
decoder.releaseOutputBuffer(outIndex, true);
}
}
/* clean */
decoder.stop();
decoder.release();
decoder = null;
}
}
I installed PTS but it doesn't work. Video playback is very fast.
Does anyone know how to set the correct presentation timestamp in this situation? Any help would be greatly appreciated.
source to share
When decoding videos with MediaCodec, you are not the one who sets the PTS, you are the one who receives the PTS. When you call releaseOutputBuffer()
with the "render" flag set, you are telling the system to render the frame as soon as possible. It is your responsibility to step by frame.
For an example of a MediaCodec video player that controls playback speed, see Grafika , specifically using <class href = "https://github.com/google/grafika/blob/master/src/com/android/grafika/SpeedControlCallback.java" rel = "nofollow"> SpeedControlCallback.
source to share
In your example, you should consider using info.mPresentationTimeUs
that received as part decoder.dequeueOutputBuffer(info, -1);
. After receiving the output frame, you must execute AV Sync
with a reference clock, which can be audio
clock or system
clock.
Please refer to this link for a good reference as an example.
source to share