GDCL News

GMFBridge 1.0.0.19 Released

Geraint Davies — Thu, 17 Jun 2010 12:19:41 GMT

I've released GMFBridge 1.0.0.19. This includes a number of bugfixes from the last few months. There are three main items:

A change to flushing on disconnect. If the graph is paused when the bridge is disconnected, we need to flush to ensure that a worker thread is not blocked inside the bridge. However, if we are at end of stream, then we know that the worker thread is not blocked and we can disconnect without flushing. This enables some single-step situations where previously the frames were being discarded by the flush.
Some DV decoders will start decoding a little before the requested start point, marking the early frames with negative timestamps. When this happens on the second or subsequent clip of a playlist, the negative timestamps prevent the two clips from being joined seamlessly. This is fixed by discarding preroll and negative timed data at a bridge change.
The GMFPreview sample now supports pause and resume of the recording without pausing the live source preview. This required a fix to the media timestamps at the bridge: these are now mapped by the same timestamp-mapping logic used for stream timestamps.

Updates to MP4 mux and demux filters

Geraint Davies — Tue, 08 Jun 2010 17:18:30 GMT

Version 1.0.0.9 of the MP4 demux includes a number of fixes accumulated over the last year. These include

Support for 'sowt' and 'twos' PCM audio in big-endian and little-endian variants in the demux
Support for files > 2GB
Improvement in performance for index lookups

I've also updated the MP4 mux to version 1.0.0.7, including a fix to H264 NAL Unit parsing and a fix for a crash on Stop if no data has been received. You can download the source and binaries here.

DirectShow for Windows Mobile

Geraint Davies — Fri, 22 May 2009 12:01:05 GMT

I've started putting together some tools for DirectShow development on Windows Mobile. This release includes a graph editor tool similar to graphedt, and some utility filters. The filters are available in source form and there are also WIN32 desktop builds of the filters.

The ViewGraph application allows you to build and view graphs on your mobile device. The user interface is similar to the desktop tool, with a few alterations to support the small screen. You can make connections by dragging from filter to filter (rather than from pin to pin) and a dialog is shown if there are multiple pins. Pin connect and render operations are also available in the menu; again, a dialog is shown if there is any ambiguity. Press and hold on either filter or connection line to bring up a context menu. The context menu includes an option to insert a filter on an existing connection.

The monitor filter is a pass-through transform that logs media types and data. All data is written to a text file and the filter supports IFileSinkFilter so that the viewgraph application prompts for an output file (for the log data).

The serialize tool includes a sink and a source filter. The sink filter saves to a file all the media type, sample data and metadata that it receives.

Go to www.gdcl.co.uk/mobile to get the tool and the filters.

GMFBridge: Fix for surface loss problem

Geraint Davies — Tue, 28 Apr 2009 09:53:59 GMT

GMBridge build 1.0.0.18 is available, with a fix for surface loss issues that have been reported by a number of people.

What I believe happens is the following:

Graph A is a capture and preview graph, containing a capture source feeding both the bridge sink and a preview renderer.
Graph B is a file writer, with the bridge source feeding a mux and the file writer filter.
During capture, when the graphs are bridged and running, a display change occurs. This can be ctrl-alt-del, or a screen resolution change, or moving the window to another monitor.
In some cases, (VMR9 windowless mode, for example), the default handling for this event is to stop and restart the graph. This will stop and restart only graph A, the preview graph, leaving graph B (file writer) still running.
When graph A starts delivering data again, the timestamps are reset to zero. However, there was no bridge disconnect or connect operation, so the bridge controller does not detect the time change.
The mux receives data timestamped at zero, and rejects the sample as out-of-order, so writing is aborted.

The problem is that bridge controller does not handle the stop and restart correctly while still bridged. I've modified the sink filter to detect a stop and restart while still bridged, and on the next data delivery it will trigger the bridge source's reconnect logic, so the timestamps will be mapped correctly.

MPEG-4 Mux & Demux 1.0.0.5 Available

Geraint Davies — Mon, 06 Apr 2009 16:45:13 GMT

6 April 2009

Increased encoder and decoder compatibility for H264 and Mpeg-4 Video

In version 1.0.0.5, the mux now accepts Byte Stream Format as well as length-preceded format for H.264, and accepts a wider range of input formats. It is able to extract decoder-specific data from the input stream if it is not supplied in the media type; this includes the VOL header for mpeg-4 and the parameter sets for H.264. With these changes, the mux and demux now work correctly with ffdshow and x264 encoders and decoders. Also, as a result, the mux project now includes some useful classes for NALU parsing. The mux also fixes a bug in which the track header dimensions were set incorrectly for larger videos.

The demux now correctly writes the CTTS table (to support B frames) even when the sample end times are not correct. This is done by calculating the frame rate from the minimum positive inter-frame interval and using the duration together with the start timestamp to calculate both decode and presentation times for each sample.

Notes on using the mpeg-4 mux

The mpeg-4 file format includes an index which lists the time and location for each access unit (picture) in the file. To build this index, the mux filter needs to know both the timestamp of each access unit and also the boundaries of each access unit. The multiplexor does not analyse individual elementary stream structures, so it needs this information to be supplied correctly in the DirectShow IMediaSample properties:

Set a timestamp on each access unit using IMediaSample::SetTime
Each IMediaSample object should contain exactly one access unit.
If the access units are too large to fit into a single buffer, you can split it access several buffers. In this case, set the timestamp only on the last buffer of the access unit. The mux will assume that the preceding buffers without timestamps are part of the same sample.
If you are using B frames, so that your timestamps are not in presentation order, it's better if you can set the end time of the sample to indicate the correct duration. Many compressed streams in DirectShow have only the start time set correctly, with the end time being start+1: in this case, without the decode timestamp or a reliable frame duration, it is difficult to calculate the CTTS table correctly.

MP4 Mux supports CTTS table for re-ordered frames

Geraint Davies — Wed, 25 Mar 2009 11:08:00 GMT

The MP4 mux now supports re-ordered frames using a CTTS table.

Timestamps in an MP4 file are derived from a list of durations. The time for each sample is simply the sum of the durations of all samples that come before it. This works fine for most types of data in MP4 files, but is not sufficient in some cases, where the frames need to be decompressed in a different order.

Many video compression schemes (such as mpeg's IPB mechanism) allow frames to refer to key frames that have not yet been displayed. For example, a typical MPEG-2 video sequence has I and P reference frames bracketing several B frames; the B frames contain only the changes from the two reference frames. This means that the decoder must decode the reference frame at the end of the group before it can decode the B frames in the middle. To allow this, the compressed frames are stored in a different order, and there are two sets of timestamps, one indicating the time to decode this frame, and the other indicating the time to present it. This frame re-ordering is supported in mpeg-4 video, but is very rarely used. It is, however, sometimes used with H.264 videos.

The MP4 sum-of-durations time gives the decode time. If the frames are not re-ordered, then this is also the presentation time. However, for re-ordered frames, an offset is needed for each frame which converts the decode time into presentation time; in an MP4 file, this offset is called the composition time offset and is contained in the CTTS table. The MP4 demux always supported this table; however, until now, the mux was unable to create it.

In DirectShow, the mux filter does not receive decode times. The timestamps attached to the IMediaSample objects are presentation times. Each sample has both a start and end presentation time, but in many multiplexing graphs, these are not very useful: the end time is either wrong (typically start+1) or it is simply derived from the start of the next sample. For this reason, the mux ignores the sample end times and creates the duration table just from the sample start times. However, in the case of frame re-ordering, it is possible to recreate the decode time from the timestamps if both start and end times are valid.

The latest version of the mux (1.0.0.4) looks at the first few sample timestamps and decides which mode to use. If both start and end times look reasonable and some of the start times are out of order, the mux will create a CTTS table. Otherwise the mux will behave as before, using just the sample start times to create only the duration table.

Updated MPEG-4 Multiplexor and Demultiplexor

Geraint Davies — Tue, 24 Mar 2009 14:10:45 GMT

I've posted an update to the MPEG4 mux and demux which includes a number of changes and bug fixes made during the course of the last year or so. This includes:

I've added support for 64-bit builds, and the binaries include release-build 64-bit DLLs.
Both mux and demux support a wider range of elementary stream types, including some uncompressed video types and byte-stream H264 (rather than length-prepended). Although the container format will support a very wide range of types, each type needs a small amount of code to translate between DirectShow media types and the container file format.
The mux now supports IAMStreamControl. This ensures that if capture is enabled after a long period of previewing with capture off, the resulting file's timestamps are calculated correctly.
The demux will now work correctly with a wider range of quicktime MOV files.
There was an embarrassing bug in the rate calculation which reversed the meaning of slowing down or speeding up.
The mux behaviour on stop has been changed. Previously, queued data was only written out if End Of Stream was received. In this version, all data received before the Stop will be written to the file.
I've improved the interleave behaviour when the audio is delivered in large buffers.

GMFBridge 64-bit build available

Geraint Davies — Mon, 16 Mar 2009 14:57:27 GMT

GMFBridge build 1.0.0.17 is available for download. This version includes x64 64-bit builds and Visual Studio 2005 project files, as well as a number of bug fixes.