How Unreal Engine picks a media player (IPStreamMedia #6)

Running a bit delayed :(

I know the one-devlog-a-week schedule has gone for a bit of a toss, my days have been super hectic these past few weeks. Anyways, onwards to the next one…

Unreal’s Media Framework

Before beginning work on the actual code of IPStreamMediaPlayer, it was imperative to understand how existing media players work in Unreal Engine. I primarily focused on three media players during my research: WmfMedia, ImgMedia and ElectraPlayer.

Unsurprisingly, I found that every media player in Unreal is a plugin - and each media player is made up of four main components: a factory, the player, an event sink and the samples. These components are generally divided into two modules, a Factory module which holds the factory obviously, and a Media Player module that holds everything else.

The factory is used to register the media player at engine startup. When OpenUrl is called in the blueprints the engine iterates through each factory’s CanPlayUrl, and CreatePlayer is called on the first one returning true. CreatePlayer is handed the engine’s event sink and it returns the new player, which keeps that sink so it can report events back to the engine.

Factory

A factory describes the media player. It defines the type of extensions and URLs the player supports, it also defines the types of media supported by the player. Additionally, the factory holds the list of platforms the player works on, its display name and its GUID.

The GUID is pretty important in our case because as it turns out WmfMedia also lists rtsp:// as a supported URL (but it doesn’t work actually ¯\_(ツ)_/¯). So to definitively ensure that we use IPStreamMedia we need to select it specifically in the UMediaSource asset which uses GUID to list all the available media players for each platform.

Why is the factory a separate module ?

You might be wondering why the factory needs to be a separate module if its job is to just hold the identity of the media player. Well it’s quite obvious once you get it - the factory holds the identity of the media player irrespective of the development platform. Imagine you are using a Windows PC for developing your Android game and you want to use a super-efficient Android media player which only works on Android platform. How would you even tell the editor that it needs to use this specific Android player if it wouldn’t even load in the editor running on Windows ? A MediaFactory helps resolve this issue, they are capable of loading on all platforms where Unreal Engine development can happen. This ensures that the developers can setup the media players correctly within the editor without worrying about their development platform choices.

Event Sink

Event Sink is the way the media players tells the Blueprints that something has happened. Events are queued and delivered on the game thread’s next tick. Without the event sink there is no way for the blueprint to know what is actually going on inside the media player. Hence if the media player has opened a URL and started receiving frames from it, blueprints won’t do anything unless the MediaOpened event has been triggered.

We’ll talk about MediaSamples later…

The payoff

After learning all of this I created the barebones of the IPStreamMedia and the IPStreamMediaFactory modules. This allowed me to create a new UMediaSource and select IPStreamMedia as my desired Windows media player. Upon calling OpenUrl, event sink fired MediaOpened which triggered a print string and on closing PIE it triggered MediaClosed which printed “Player Closed”.

The next step is to join this part with the FFmpeg baloney. Until then, Cheers 🍻!