All you need to know about video codecs, containers and formats

In this guide, we’ll break down the fundamentals of videos:

• A primer about codecs, containers and formats
• Video codecs usage in web development
• Codecs recommendations for the web

Video codecs, containers and formats

Codecs are responsible for compressing and decompressing digital video content. Being in charge of the encoding and decoding processes, codecs play a key role in performance and compatibility. You can think of a container and a file format interchangeably. A single file format (or container) can support and contain the data produced by a variety of different codecs.

The file format determines the extension of the physical file we see on our devices, such as .avi, .mp4, .webm, and .mov. In fact, video containers can contain more than just video information, like audio streams, subtitles, menus, and more metadata. This information can be used to deliver richer viewing experiences.

For example:

• Audio Video Interleaved (.avi) can contain video data from MPEG-2, DivX, and DV (digital video) codecs
• MPEG-4 (.mp4) is most commonly used for the h.264 codec while WebM files use video streams compressed by the VP8 or VP9 video codec

To summarize: containers are compatible with a given list of video codecs; the words formats and containers are often used interchangeably.

Video codecs usage in web development

Now that we’ve got a better understanding of the basics, let’s take a look at the most commonly used codecs on the web.

H.264/AVC (Advanced Video Coding)

H.264 is widely regarded as the most adapted codec for web use and online video streaming. Being continuously developed and consisting of a number of different encoding profiles it is very flexible and supported on a wide range of devices. For these reasons, it’s considered a family of standards and not just a single codec.

Many popular file formats use the H.264 codec, including MPEG-4 (.mp4), QuickTime (.mov), Flash (.f4v), and 3GP (.3gp).

H.265/HEVC (High-Efficiency Video Coding)

H.265 is the next generation of the H.264 standard and is set to replace it in the near future. The developers – which developed both standards – state that H.265 can deliver twice the video quality at the same bitrate.

Like H.264, H.265 is supported by the .mp4, .m4v, .mov, .mkv, and other containers. It aims to become the go-to format for high-resolution video, like 4K streaming. But as of today, H.265 doesn’t enjoy widespread support outside of Safari.

VP9

VP9 is an open-source video codec developed by Google to supersede VP8. Customized for videos with resolutions above 1080p, it delivers similar performance improvements than H.265 over H.264, while outperforming H.265 for some content higher than HD levels.

VP9 is a very widely used standard today and is supported by most browsers and mobile devices – but not by the Apple ecosystem. The .webm container is the most popular file format using VP9 encoding.

AV1

VP10 was engineered by Google to be the successor of VP9 but was incorporated into the AV1 (AOMedia Video 1). AV1 is still an open-source, royalty-free video codec and is supported by the .mkv, .webm, ISO base media file format (.3gpp), and WebRTC containers.

AV1 still lacks optimization for encoding and decoding, operating much slower than most of its counterparts. However, it does offer highly efficient bitrates, providing savings over VP9, HEVC, and AVC, especially for ultra-HD video content at 4K or 8K. It is mainly used for streaming content.

Codecs recommendations for the web

So, is there a “best” video codec? As is usually the case for web performance, there is no one-size-fits-all answer. Standard maturity and browser support are just as important as the codec performance itself.

While H.264 and VP9 are already established efficient codecs with nearly universal support, decreasing page loading time remains a priority to create better online experiences and increase conversion. To that end, we encourage you to always serve the most efficient codec where possible.

Next-gen codecs like H.265 comfortably beat existing ones on paper, but they currently suffer a lack of support in the ecosystem. To gain the most out of them, we recommend using context-aware generation of codecs and formats to tailor the experience to each user.

Closing thoughts

Wrapping your head around codecs and formats can be a challenge. But as video content becomes increasingly prominent online, it is important to understand the fundamentals of its impact on web performance and user experience.

As videos prove to improve engagement and increase ecommerce conversion, it is time for website owners to embrace videos as a competitive advantage to engage with their visitors. Context-aware optimization becomes pivotal to reaping those benefits while ensuring maximum compatibility.