Clever Geek Handbook

Sound equivalent of SPS and PPS for multiplexing MPEG-TS Appendix B? What is "DecoderInfo"?

I am using Bento4 library to multiplex an Application B TS (MPEG-2 Transport Stream) file with my h264 and AAC videos that are generated from VideoToolbox and AVFoundation respectively, as input to an HLS (HTTP Live Streaming) streaming stream. This question is not necessarily Bento4 specific: I am trying to understand the basic concepts so that I can accomplish the task, preferably using Apple libs.

So far I have figured out how to create AP4_AvcSampleDescription

by choosing from my CMVideoFormatDescriptionRef

various data types and most importantly creating SPS and PPS using index 0 and 1 respectively so CMVideoFormatDescriptionGetH264ParameterSetAtIndex

that I can just insert byte buffers into Bento4. Great that I need all the header information so I can request Bento4 to mux the video into a ts file!

Now I am trying to connect audio to the same file. I use mine CMAudioFormatDescriptionRef

to get the information I need to build mine AP4_MpegAudioSampleDescription

, which Bento4 uses to create the necessary QT atoms and headers. However, if the fields are a byte buffer "information decoder", no explanation of what it is, or code to generate one of the data. I would hope to have CMAudioFormatDescriptionGetDecoderInfo

or something else, but I cannot find anything like it. Is there such a function in any Apple library? Or is there a good spec I haven't found on how to generate this data?

Or, on the contrary, am I on the wrong path? Is there an easier way to mux ts files from Mac / iOS database?

+1


source to share


2 answers


Multimedia audio in MPEG-TS is surprisingly simple and doesn't require a complex title like a video stream! Each fetch buffer only requires a 7-byte ADTS header before writing it as PES.

Bento4 uses the DecoderInfo buffer to parse it into an instance AP4_Mp4AudioDecoderConfig

so that it can extract the information needed for the ADTS header. Instead of bypassing this information, I made a copy AP4_Mpeg2TsAudioSampleStream::WriteSample

that writes a CMSampleBufferRef

. It can be easily generalized to other audio frameworks, but I'll just paste it as is for reference:



// These two functions are copy-pasted from Ap4Mpeg2Ts.cpp
static unsigned int GetSamplingFrequencyIndex(unsigned int sampling_frequency) { ... }
static void
MakeAdtsHeader(unsigned char *bits,
               size_t  frame_size,
               unsigned int  sampling_frequency_index,
               unsigned int  channel_configuration) { ... }

static const size_t kAdtsHeaderLength = 7;

- (void)appendAudioSampleBuffer2:(CMSampleBufferRef)sampleBuffer
{
    // Get the actual audio data from the block buffer.
    CMBlockBufferRef blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer);
    size_t blockBufferLength = CMBlockBufferGetDataLength(blockBuffer);

    // Get the audio meta-data from its AudioFormatDescRef
    CMAudioFormatDescriptionRef audioFormat = CMSampleBufferGetFormatDescription(sampleBuffer);
    const AudioStreamBasicDescription *asbd = CMAudioFormatDescriptionGetStreamBasicDescription(audioFormat);

    // These are the values we will need to build our ADTS header
    unsigned int sample_rate = asbd->mSampleRate;
    unsigned int channel_count = asbd->mChannelsPerFrame;
    unsigned int sampling_frequency_index = GetSamplingFrequencyIndex(sample_rate);
    unsigned int channel_configuration = channel_count;

    // Create a byte buffer with first the header, and then the sample data.
    NSMutableData *buffer = [NSMutableData dataWithLength:kAdtsHeaderLength + blockBufferLength];
    MakeAdtsHeader((unsigned char*)[buffer mutableBytes], blockBufferLength, sampling_frequency_index, channel_configuration);
    CMBlockBufferCopyDataBytes(blockBuffer, 0, blockBufferLength, ((char*)[buffer mutableBytes])+kAdtsHeaderLength);

    // Calculate a timestamp int64 that Bento4 can use, by converting our CMTime into an Int64 in the timescale of the audio stream.
    CMTime presentationTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer);
    AP4_UI64 ts = CMTimeConvertScale(presentationTime, _audioStream->m_TimeScale, kCMTimeRoundingMethod_Default).value;

    _audioStream->WritePES(
        (const unsigned char*)[buffer bytes],
        (unsigned int)[buffer length],
        ts,
        false, // don't need a decode timestamp for audio
        ts,
        true, // do write a presentation timestamp so we can sync a/v
        *_output
    );
}
+1


source


The "decoder" byte buffer required by Bento4 to instantiate AP4_MpegAudioSampleDescription is codec initialization data that is codec specific. For AAC-LC audio, this is usually 2 bytes of data (for HE-AAC you will get a few more bytes), the details of which are in the AAC spec. For example, 44.1 kHz, stereo, AAC-LC streams will have [0x12.0x10] as their initialization data. Most of Apple's APIs pass this type of codec initialization data through what they call "Magic Cookies." Probably the CMAudioFormatDescriptionGetMagicCookie function will return what you need.



0


source






More articles:


All Articles