Customising Graph Building with IStreamBuilder

DirectShow’s automated graph building works surprisingly well most of the time, but there are cases where you need to customise the graph-building logic a little. Doing this when you are building the graph in your application is quite straightforward (one of the simplest is to insert a preferred filter into the graph before calling RenderFile). But from within a filter, it’s quite a bit harder.

Consider a filter that outputs subtitles. By default, the graph manager will render this output pinto a separate renderer filter. However, you really want it connected to a secondary pin on the VMR, so that the subtitles are overlaid correctly on the video. How can you do this from within your filter, so that subtitles will be rendered even in other apps, such as Windows Media Player?

The answer is to implement IStreamBuilder, which allows you to control the rendering of your output pin. I’ve used this on several occasions, and only recently discovered a couple of bugs in the DirectShow implementation. First, I’d like to outline the correct use of IStreamBuilder before describing the bugs.

Rendering a pin

When DirectShow is trying to render a pin, it will query the pin for the IStreamBuilder interface. This is true for calls to RenderPin as well as RenderFile, but not for Connect operations (so you can control the rendering of your subtitle output without getting in the way when an app wants to connect your pin itself).

If the pin implements IStreamBuilder, the graph manager will call its Render method. In your Render method, you can add filters, connect and disconnect pins, and most importantly, you call delegate back to the graph manager without an infinite loop. So, for example, some demux filters will implement IStreamBuilder so that they can add a preferred decoder filter to the graph, and then call the graph manager’s Render method to complete the rendering.

Backout

If you implement Render, you must also implement the Backout method to undo everything you did in Render. This includes removing any filters you added and undoing any connect or disconnect operations. Since the graph manager backs out in reverse order, you should find that when your Backout method is called, the graph state is the same as it was at the end of your Render method. In fact, this means that it is safe to mess with connections that the graph manager has made, provided that you undo these operations in your Backout method.

DirectShow Bugs

I was prompted to write this note by the recent discovery of a couple of bugs in the DirectShow handling of IStreamBuilder.

While the graph manager is working down the graph rendering pins, it keeps track of what fraction of the stream each pin represents. So, if the demux has two outputs (for video and audio), each output pin represents 50% of the original source. If the video decoder then splits into a video and a subtitle output, then each of those is 25% of the output (whereas the single audio output is still 50%). If a particular configuration of filters is found that renders some outputs but not others, the graph manager keeps a note of what fraction of the stream is rendered. If it fails to find any configuration that renders all the outputs, it will go back to the configuration that rendered the highest fraction, and rebuild that graph, before reporting a “partial success” code.

When recreating a previous “best so far” configuration, the graph manager pulls the same instance of the filter from its cache, and reconnects its input. It will then attempt to repeat the previous render operations on the output pins. Now, sometimes, the filter will not create the same output pins when it is reconnected. In my case, a problem with cleaning up in BreakConnect meant that no audio pins were created when the demux filter was connected a second time.

In the regular Render code, this is detected and results in a simple error return. But when the graph builder is using IStreamBuilder to render an output pin, the code is in a slightly different order and a missing pin causes DirectShow to crash. Since the crash in deep in the filter graph manager, it’s very hard to relate this to the bug in the filter that caused it.

There’s another problem in this same code, unfortunately. The graph manager code to recreate a partially-successful graph doesn’t work when IStreamBuilder is used. The pin render failure that should result in a partial-success code actually ends up being returned as an E_FAIL, so the whole render fails. So if you implement IStreamBuilder for all your demux output pins, and two of them can be rendered, but the third fails, you will not get a partial graph built in this case, as you would normally.