Android's Codec Support

If you’ve been exposed to media playback on Android, you will have quickly discovered that not all devices are equal. The capabilities of a device comes down to a number of factors, which include the version of Android as well as what the manufacturer included. Whenever we talk about encoding or decoding on Android, we always refer to the MediaCodec API. This is the low level codec (the component responsible for decoding or encoding data) that is part of Android’s multimedia support infrastructure, and provides different capabilities depending on the device. In this short article, we’ll explore how to find what a device can support and what it’s capable of.

The first thing you’ll probably be asking is: how can you easily find what codecs your device can handle? Arguably the easiest way to do this is to install an app like Media Codec Info and see what it says, but where’s the fun in that? Due to the variance in device support, it made sense for Android to provide an easy mechanism for manufacturers to configure the device via a XML file located as part of the installation. This root file is called media_codecs.xml that itself can reference other XML files located in the same directory. This file can be found as follows:

[mbp-ianb ~]% adb shell

redfin:/ $ ls /vendor/etc/media_codecs*.xml
                                                                                              
/vendor/etc/media_codecs.xml  /vendor/etc/media_codecs_aosp_c2.xml  /vendor/etc/media_codecs_c2.xml  /vendor/etc/media_codecs_performance.xml  /vendor/etc/media_codecs_performance_c2.xml

Let’s take a quick look at what this file may contain…

<Included>
    <Decoders>
        <!-- C2 decoders -->
        <MediaCodec name="c2.qti.avc.decoder" type="video/avc">
            <Alias name="OMX.qcom.video.decoder.avc" />
            <Limit name="size" min="96x96" max="4096x2160" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="16x16" />
            <Limit name="bitrate" range="1-120000000" />
            <Limit name="frame-rate" range="1-480" />
            <Feature name="adaptive-playback" />
        </MediaCodec>
        <MediaCodec name="c2.qti.avc.decoder.low_latency" type="video/avc">
            ...
    </Decoders>

    <Encoders>
        <!-- C2 encoders -->
        <MediaCodec name="c2.qti.avc.encoder" type="video/avc">
            ...
    </Encoders>
</Included>

There are a few things worth noting here:

• There are two separate lists for encoders and decoders
• The mime type is telling Android what compression type it can handle
• The name of the MediaCodec is what Android will use to instantiate it (when required)
• The capabilities of the codec is defined as part of its description. This can include details about minimum/maximum bitrate, supported resolutions, etc.
• A device may have multiple codecs for the same mime type, each having different capabilities

Now that we understand what a MediaCodec is, and how it’s configured on the device, let’s explore how we would use this within an Android application. Thankfully, we won't be doing any XML parsing ourselves. Android included an API which exposes all this information natively, known as the MediaCodecList. Using this API, you can either iterate over all available codecs or use the findDecoderForFormat function to find a suitable one for a given MediaFormat. The MediaFormat defines the mime type as well as some optional fields, such as bitrate, resolution, etc. This additional format-specific data can be used to compare the capabilities of the MediaCodec to confirm support.

There are unfortunately a few issues when it comes to device specific codecs, and unfortunately not all MediaCodecs work the same or even work at all. As an application developer, we can’t really be expected to work around these issues and thankfully, we (mostly) don’t need to. Back in 2014, Google released ExoPlayer, an open source media player library. Whilst it is probably all you need as an application developer, if you are looking to do some custom MediaCodec handling, it’s an incredibly useful reference. Take the above example of simply iterating over codecs. We could use the mentioned MediaCodecList, but we could also use ExoPlayer’s MediaCodecUtil.getDecoderInfos. You may ask what advantage this alternative implementation gives us, and the answer lies in one of its internal methods isCodecDecoderUsable. This is effectively a denylist of codecs for specific devices. Google has identified, and continues to identify devices with problematic codecs and improve their MediaCodec selection. Using this implementation will go some way to avoid exposing yourself to device specific issues.

There is lots more to understand about MediaCodecs on Android, and unfortunately even more gotchas. If people are interested, I'll follow up on this area in a future article. Let me know!