GDCL Home / mpeg4
|
|
|
|
||
|
GDCL Home / mpeg4
|
|||||
The GDCL Mpeg-4 Demultiplexor and Multiplexor filters are now freely available for download in source form. You can re-use them in your projects, commercial or otherwise (provided that you don't pretend that you wrote them) or use them as sample code when starting on your own project. Of course support and documentation are somewhat limited.
This initial release is being made available because of the insolvency of a customer. As I get time, I intend to enhance these filters. For now, they support ISO Mpeg-4 files containing Mpeg-4 and H.264 video and a variety of audio formats. The files created by the multiplexor will work with QuickTime and the iPod.
Published: September 2006. Latest Update: April 2009.
Download source and binaries here.
In version 1.0.0.5, the mux now accepts Byte Stream Format as well as length-preceded format for H.264, and accepts a wider range of input formats. It is able to extract decoder-specific data from the input stream if it is not supplied in the media type; this includes the VOL header for mpeg-4 and the parameter sets for H.264. With these changes, the mux and demux now work correctly with ffdshow and x264 encoders and decoders. Also, as a result, the mux project now includes some useful classes for NALU parsing. The mux also fixes a bug in which the track header dimensions were set incorrectly for larger videos.
The demux now correctly writes the CTTS table (to support B frames) even when the sample end times are not correct. This is done by calculating the frame rate from the minimum positive inter-frame interval and using the duration together with the start timestamp to calculate both decode and presentation times for each sample.
The mpeg-4 file format includes an index which lists the time and location for each access unit (picture) in the file. To build this index, the mux filter needs to know both the timestamp of each access unit and also the boundaries of each access unit. The multiplexor does not analyse individual elementary stream structures, so it needs this information to be supplied correctly in the DirectShow IMediaSample properties:
Timestamps in an MP4 file are derived from a list of durations. The time for each sample is simply the sum of the durations of all samples that come before it. This works fine for most types of data in MP4 files, but is not sufficient in some cases, where the frames need to be decompressed in a different order.
Many video compression schemes (such as mpeg's IPB mechanism) allow frames to refer to key frames that have not yet been displayed. For example, a typical MPEG-2 video sequence has I and P reference frames bracketing several B frames; the B frames contain only the changes from the two reference frames. This means that the decoder must decode the reference frame at the end of the group before it can decode the B frames in the middle. To allow this, the compressed frames are stored in a different order, and there are two sets of timestamps, one indicating the time to decode this frame, and the other indicating the time to present it. This frame re-ordering is supported in mpeg-4 video, but is very rarely used. It is, however, sometimes used with H.264 videos.
The MP4 sum-of-durations time gives the decode time. If the frames are not re-ordered, then this is also the presentation time. However, for re-ordered frames, an offset is needed for each frame which converts the decode time into presentation time; in an MP4 file, this offset is called the composition time offset and is contained in the CTTS table. The MP4 demux always supported this table; however, until now, the mux was unable to create it.
In DirectShow, the mux filter does not receive decode times. The timestamps attached to the IMediaSample objects are presentation times. Each sample has both a start and end presentation time, but in many multiplexing graphs, these are not very useful: the end time is either wrong (typically start+1) or it is simply derived from the start of the next sample. For this reason, the mux ignores the sample end times and creates the duration table just from the sample start times. However, in the case of frame re-ordering, it is possible to recreate the decode time from the timestamps if both start and end times are valid.
The latest version of the mux (1.0.0.4) looks at the first few sample timestamps and decides which mode to use. If both start and end times look reasonable and some of the start times are out of order, the mux will create a CTTS table. Otherwise the mux will behave as before, using just the sample start times to create only the duration table.
I've posted an update to the MPEG4 mux and demux which includes a number of changes and bug fixes made during the course of the last year or so. This includes:
I've found that the mpeg-4 demux does not work on systems with Pinnacle Studio installed. This is because Studio registers a custom source for mpeg-4 (based on code from dicas) and also registers a custom type for the file which is not MEDIATYPE_Stream. To fix this, I've made a minor change to the demux input pin to accept any media type, even non-stream. Additionally, you will need to explicitly insert File Source (Async) and render its output pin instead of using RenderFile if you want to use this demux on these systems.
I've released a minor update fixing a couple of small (one-line) bugs. The demultiplexor did not handle fixed-size samples correctly (e.g. PCM audio). The multiplexor supported some DivX fourccs as inputs (DIVX, divx) but not all of them (DX50) so I've added those.
Note: both mux and demux filters need a small amount of code added for each media type that is to be supported. So far I have only added a limited set of types. If you need other types to be accepted, please get in touch directly It's likely that only a few lines of code will be needed.