Android: How to use MediaMuxer with video / mp 4v-es instead of video / avc?
I want to use mp4v-es instead of avc on some devices. The encoder works fine using avc, but when I replace it with mp4v-es muxer reports:
E/MPEG4Writer(12517): Missing codec specific data
as in MediaMuxer error "The multiplexer could not be stopped" and the video cannot be played. The difference is that I add the correct format / format to the muxer without getting any error:
...else if (encoderStatus == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
MediaFormat newFormat = encoder.getOutputFormat();
mTrackIndex[encID] = mMuxer.addTrack(newFormat);
Is there a difference in handling mp4v-es compared to avc? One mention, I just skip "bufferInfo.flags and MediaCodec.BUFFER_FLAG_CODEC_CONFIG" when this happens, as avc didn't need it. Thank.
source to share
As Ganesh showed, unfortunately it looks like this is not possible now, without changing the platform source.
There are actually two ways of passing codec-specific data into the inner MPEG4Writer class, but neither works unchanged.
As Ganesh found out, the logic for remapping the MediaFormat keys to the internal format does not seem to be able to handle specific codec data for any other video codec than H264. A tested modification that fixes this issue is as follows:
diff --git a/media/libstagefright/Utils.cpp b/media/libstagefright/Utils.cpp
index 25afc5b..304fe59 100644
--- a/media/libstagefright/Utils.cpp
+++ b/media/libstagefright/Utils.cpp
@@ -549,14 +549,14 @@ void convertMessageToMetaData(const sp<AMessage> &msg, sp<MetaData> &meta) {
// reassemble the csd data into its original form
sp<ABuffer> csd0;
if (msg->findBuffer("csd-0", &csd0)) {
- if (mime.startsWith("video/")) { // do we need to be stricter than this?
+ if (mime == MEDIA_MIMETYPE_VIDEO_AVC) {
sp<ABuffer> csd1;
if (msg->findBuffer("csd-1", &csd1)) {
char avcc[1024]; // that oughta be enough, right?
size_t outsize = reassembleAVCC(csd0, csd1, avcc);
meta->setData(kKeyAVCC, kKeyAVCC, avcc, outsize);
}
- } else if (mime.startsWith("audio/")) {
+ } else if (mime == MEDIA_MIMETYPE_AUDIO_AAC || mime == MEDIA_MIMETYPE_VIDEO_MPEG4) {
int csd0size = csd0->size();
char esds[csd0size + 31];
reassembleESDS(csd0, esds);
Second, instead of passing codec-specific data like csd-0
in MediaFormat, you could in principle pass the same buffer (with the flag set MediaCodec.BUFFER_FLAG_CODEC_CONFIG
) to MediaMuxer.writeSampleData
. This approach does not work currently, as this method does not check the codec config flag at all - it can be fixed with this modification:
diff --git a/media/libstagefright/MediaMuxer.cpp b/media/libstagefright/MediaMuxer.cpp
index c7c6f34..d612e01 100644
--- a/media/libstagefright/MediaMuxer.cpp
+++ b/media/libstagefright/MediaMuxer.cpp
@@ -193,6 +193,9 @@ status_t MediaMuxer::writeSampleData(const sp<ABuffer> &buffer, size_t trackInde
if (flags & MediaCodec::BUFFER_FLAG_SYNCFRAME) {
sampleMetaData->setInt32(kKeyIsSyncFrame, true);
}
+ if (flags & MediaCodec::BUFFER_FLAG_CODECCONFIG) {
+ sampleMetaData->setInt32(kKeyIsCodecConfig, true);
+ }
sp<MediaAdapter> currentTrack = mTrackList[trackIndex];
// This pushBuffer will wait until the mediaBuffer is consumed.
As far as I can tell, there is no way to have MPEG4 MPEG4 videos with MediaMuxer right now using the public API without changing the platform source. Given the issues in Utils.cpp above, you cannot mux any video format that requires specific codec data, except H264. If VP8 is an option, you can mux it into webm files (along with audio vorbis), but hardware encoders for VP8 are probably much less common than hardware encoders for MPEG4.
source to share
I am assuming that you have the ability to change sources Stagefright
, and hence I have a suggested solution for your problem, but it requires customization.
Background:
When encoder
encoding completes, the first buffer will have information csd
, which is usually marked with OMX_BUFFERFLAG_CODECCONFIG
. When such a buffer is returned to MediaCodec
, it must store the same as csd-0
in MediaCodec::amendOutputFormatWithCodecSpecificData
.
Now , when this buffer is assigned MediaMuxer
, it is treated as the part addTrack
in which convertMessageToMetadata
. If you refer to its implementation, we may notice that for is video
processed only AVC
and is used by default audio
for ESDS
.
EDIT:
Here's my recommendation - change this line as shown below and try my experiment
}
if (mime.startsWith("audio/") || (!strcmp(mime, MEDIA_MIMETYPE_VIDEO_MPEG4)) {
With this change, I feel like it should work for the video track too MPEG4
. The change is to convert else if
to if
, because the previous check for video
will also try to process the data, but only for AVC
.
source to share