Streaming Codecs for Video and Audio: What Broadcasters Need to Know in 2024

As a broadcaster, you should understand the technology behind your broadcasting. More specifically, it’s a good idea to know what codecs are and the role they play.

Codecs affect your streaming equipment needs, compatibility, budgets, bandwidth needs, video, quality and more. Moreover, using the best codecs available can keep you on the cutting edge of what’s visually possible and advance your career.

In this article, we cover everything you need to know about codecs. We start with the basics, then move on to common video and audio codecs. We then delve deeper and discuss bandwidth and next-generation codecs. Finally, we provide some expert recommendations on codec settings for live streams.

This post covers a topic that is subject to regular new developments. This post reflects the most accurate information as of April 2024.

Table of Contents:

• Streaming Codecs: The Basics
• Common Audio and Video Codecs
• H.264 Profiles
• Wrappers
• Transport Protocol
• Codecs and Bandwidth
• Next-Generation Streaming Codecs
  • 1. HEVC, or H.265:
  • 2. AV1:
  • 3. Opus
• Recommended Codec and Settings for Live Streaming
• Conclusion

Streaming Codecs: The Basics

When you record a video, RAW digital video files are generated. RAW digital video files are composed of a series of still pictures. When these videos are played in succession at a high speed, you get a video.

Similarly, RAW digital audio is simply a collection of measurements of the magnitude of various tones. When played together, you get understandable audio.

This RAW data is rich in information. However, that also means it’s bulky and unwieldy. A video composed of thirty images per second quickly becomes extremely large. Professional-grade cameras that capture RAW video can easily create terabytes of data from a single day of shooting. The same is true of RAW audio recorders.

While this is valuable in certain industries, it’s a liability for online video and live streaming. Viewers’ internet connections, mobile devices, and home computers simply aren’t powerful and fast enough to process this data. Storage and bandwidth are expensive, and RAW files tear through them.

Codecs offer a solution to this file size and speed issue. The term codec stands for “coding/decoding.” A codec is a mathematical process that reduces digital file sizes by “throwing out” or reducing unnecessary data. The complex part is doing this intelligently while retaining as much quality as possible. That is what codec video is.

Different codecs have different ways of doing this. Codecs compare each frame to the previous frame. If data remains the same (for example, a dark corner of the frame in both images), it can be tossed from the second frame. This is a simplified example, but it helps you understand how a codec works

Over time, the methods have improved. However, codecs that compress video to smaller sizes while retaining more quality tend to require more processing power—both to encode and decode.

As we mentioned, there are many different codecs available. There are a few main codec types that are commonly used for online audio and video.

Some common audio and video codecs include:

H.264/AVC

The most used codec and encoding output used today are H.264 files. It is also referred to as Advanced Video Coding (AVC). This we developed by the Telecommunications Union and the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group.

H.264 has become so popular because it isn’t just used for streaming; it is used for Blu-ray disks and cable broadcasting. This is more than likely due to who developed it.

Plus, H.264 often works together with AAC audio codec and can work with .mp4, .mov, .F4v, .3GP, and .ts containers.

The thing that makes H.264 so popular is that it can be used on virtually any device – any browser, any device – and can play H.264 videos. It is widely supported, and publishers know how to use it.

According to Bitmovin’s 2022 Video Developer report, over 91% of those who took the survey use H.264.

The downside to this is that it doesn’t work well with 4K video or HDR content. It works well with low-latency streaming. It is commonly used with HTTP-based and WebRTC-based applications. This format has been around the longest and looks to stay that way for a while, as it can work with virtually any device.

V9P

Google developed the V9P as an open-source, royalty-free alternative to H.264. It was released in 2013.

Since being released, support for VP9 has increased. It is supported on:

• YouTube
• Chrome browser
• Android phone
• Mozilla’s Firefox browser
• Apple’s Safari browser
• All new iOS devices
• Samsung, Sony, LG, Roku TVs

More than 90% of Chrome-encoded WebRTC video uses either VP9 or VP8, its predecessor. It ranks 2nd behind H.264/AVC codec regarding accessibility on devices. This is largely driven by Google using it for YouTube and Netflix’s adoption of this codec.

It is a better version of AV1, which we will review below, as more devices support this codec. It is a higher-quality compression technology than H.264/AVC and is more compatible with devices than most other H.264 alternatives.

Plus, it works well with 4K streaming.

H.265/HEVC

The SO/IEV Moving Picture Experts Group made H.265 a successor to H.264. It is also called High-Efficiency Video Coding.

The big difference from H.264 is that it is made to create smaller files, uses less bandwidth, and supports high-resolution streaming. Although this is designed as a successor to H.264, it is only used by about 10% of encoded files.

There has been some confusion about its royalties and what developers will have to pay for using the codec, which has stopped its widespread adoption. Royalty confusion helped spur the development of the AV1 codec.

It is a good codec to use if you want to deliver 4K or HDR content.

AV1

Major tech players Amazon, Netflix, Cisco, Microsoft, Google, and Mozilla came together to form the Alliance for Open Media. They then created AV1, which is a royalty-free and open-source alternative.

It is designed to be more efficient than H.264 and H.265. It works well with high-quality content but has long encoding times, making it more expensive to use.

H.266/ VVC

H.266, also referred to as Versatile Video Coding, or VVC was just created in 2020. It is made to use instead of H.264 and H.265; however, royalty issues impede its widespread adoption. Confusion around royalties has stopped its widespread adoption.

It is made to provide bandwidth savings while delivering high-quality content.

ACC

ACC is regarded as the best audio codec that you can use. It was made in the late 1990s. Even though it was made decades ago, it still creates great audio quality using low bitrates.

It is also widely compatible with all devices, which is why it continues to be the most used audio codec.

MP3

The original codec that helped to drive media sharing on the internet started with MP3. MP3 isn’t used as a codec anymore; most people identify it as a file type. ACC is so much more effective.

H.264 Profiles

Since H.264 is the most popular video codec for online video, it’s worth mentioning a key element of it: profiles. Profiles are different versions of the H.264 codec. There are many different H.264 profiles, but the most common are Baseline, Main, and High.

Baseline and Main were created for low-powered devices. However, today the High profile is an excellent choice for most live video broadcasts. High-profile H.264 is the best codec for streaming video live.

Wrappers

Let’s clear up one common confusion. The file extension at the end of a digital file often doesn’t tell you anything about the codec. Common filenames for media end in .mp4, .mov or .avi.

These are called “wrappers” or “container formats.” A wrapper allows a file to contain multiple different elements, such as subtitles and metadata. The container format you use isn’t particularly important for online videos.

If in doubt, use .mp4. It’s the most widely supported format in wide use today. mp4 is the best video format for web streaming, as it will work with most other tools necessary for video streaming.

Transport Protocol

As you can tell, internet video requires a series of technologies built on top of each other to solve new challenges. The transport protocol falls into this category.

Transport protocols are methods for streaming multimedia over the internet. They must be understood by the sender and receiver to function.

There are two very common protocols in use today in the live-stream industry. The first is the HLS streaming protocol, which is what we use at Dacast to deliver streams to viewers. HLS is widely compatible and provides good performance.

The second protocol is RTMP (real-time messaging). This is an aging standard, but it’s still widely used for stream ingestion

Protocols are regularly updated as technology advances, and MPEG-DASH is on the rise.

Codecs and Bandwidth

As you’ve learned, codecs help compress files to make them easier to transmit over the internet. Internet speeds are increasing, but video files are getting bigger as well. 4K video is becoming more common, as are HDR and HFR Each of these advances in video technology requires more space.

The size and quality of your video stream determine how much bandwidth you use. Higher bitrates (which correspond to higher quality) require more bandwidth to send.

You’ll need to ensure your internet connection is fast enough to stream at your desired bitrate. You may also want to ensure your viewers have a quality viewing experience by streaming in multiple bitrates at once. Multiple bitrates allow the quality of the video to adjust to fit your viewer’s internet speeds and device abilities.

Next-Generation Streaming Codecs

Your bandwidth usage is dependent on the codec you use. There is a variety of next-generation video and audio codecs in development today.

Let’s mention a few briefly.

1. HEVC, or H.265:

H.265 is the successor to H.264. It promises to deliver video with about double the quality/efficiency ratio of H.264. It is the future of streaming codecs.

2. AV1:

Another forthcoming codec to check out is AV1. This open-source alternative to H.265 provides better quality and may be ubiquitous in a few more years. It is a strong contender to become a top video codec in years to come.

3. Opus

Opus is an up-and-coming audio codec. The Opus codec delivers better audio quality at every bitrate compared to other audio codecs. Opus codec is free and open-source and natively supported on iOS and Android.

Recommended Codec and Settings for Live Streaming

As we mentioned, codecs are an ever-changing technology, so the “recommended” codecs change relatively quickly.

For now, we recommend that most broadcasters use the H.264 codec for video and the AAC codec for audio. These are the best streaming codecs for live and on-demand video content delivery.

These recommendations will likely change in the future as codec technology develops, but as of right now, these are the best choices to reach the maximum possible audience with your OTT video

For other encoder settings such as bitrate, resolution, sample rate, and so on, check out our blog post that covers recommended encoding settings

Conclusion

With this new knowledge of codecs and streaming, you can now improve your broadcasting. Understanding the technology means you can use it to its fullest potential. Furthermore, recognizing how quickly this technology shifts and knowing the next generation of codecs means you can remain at the apex of video broadcasting.

Now that’re equipped with this knowledge, it’s time to pair yourself with a professional platform. Dacast is a live-streaming and broadcasting solution fit for organizations of all shapes and sizes. With Dacast, you can broadcast your video securely in the highest quality. And with advanced features such as monetization, video transcoding, and live-stream recording, there’s no greater all-round solution.

You can try Dacast today by signing up for our 14-day risk-free trial.

Get started for free   

Any questions, comments, or ideas? Let us know in the comments section, below! And for regular tips on live streaming and exclusive offers, feel free to join our LinkedIn group.