Safe place to truncate H.264 data?
I am writing a transmixer to convert MPEG-TS files from an HLS stream to MP4 files that can then be played in a web browser.
I expected to be able to just map TS files to MP4 files one time, but it turns out that some HLS streams are split so that individual TS files don't always start with IDR frames; this results in frozen images and stuttering whenever you cross a segment boundary.
I suppose I can fix this by concatenating data from multiple TS files, ignoring the original file boundaries, self-identifying IDR frames in the video stream, and running each new MP4 segment on an IDR frame. However, if I do this, I am worried about possible corruption at the end of the files; IDR frames ensure that no later frame can refer to any earlier data, but there is nothing to indicate that early B-frames cannot look forward on the IDR frame.
So how do I know where it is safe to make the cut so that the B-frames before the cut don't try to look past it?
source to share
I think you have no problem if you finish cutting the IDR frame. But I'm not 100% sure.
From the H.264 spec and for example RFC3984 :
IDR image: An encoded image containing only the I or SI fragments that are "reset" during the decoding process. After decoding an IDR picture, all of the next coded pictures in the decoding order can be decoded without inter-prediction from any picture decoded to an IDR picture.
Note that frames cannot refer to other frames before IDR in decoding order .
For the B-frame to refer to any frame after the IDR frame, the associated frame must appear earlier in the file, as it must be decoded first. Thus, the B frame must be after the IDR frame. This would mean that this B-frame cannot refer to any frame before the IDR frame. It doesn't make any sense to me.
For example: suppose the following frames are in display order:
(other frames) B IDR P
if P refers to IDR and B will refer to P, this should be the decoding order:
(other frames) IDR P B
In this example, B cannot refer to any of the other frames, it can only refer to IDR and P.
Note that even if this happens, you might find it if you find a frame after the IDR, which is in the display order before the IDR.
source to share