MPEG file format honored with prestigious Emmy® Award

On 25 January, 2021, the New York-based National Academy of Television Arts & Sciences (NATAS) announced that the MPEG file format subgroup will be honored with a Technology and Engineering Emmy® Award in 2021 for the standardization of the ISO base media file format, which is used worldwide for playback and delivery of movies and television programs. The format is the foundation of a family of multimedia standards, including MP4 and 3GP files supported by billions of phones and computers, and the DASH streaming standard used by Netflix and YouTube to name a few.

This is not the first time MPEG technology has been recognized with an Emmy®. In 2017, I was present when MPEG and the Video Coding Experts Group of ITU-T received a Primetime Engineering Emmy® Award for High Efficiency Video Coding (HEVC). Before that, MPEG-1, MPEG-2, and Advanced Video Coding (AVC) had also been recognized with Emmy® Awards. Who said standardization isn’t glamorous?

Since the first version of the ISO base media file format was standardized (as MP4) in 1999, many companies have contributed to its success. Ericsson has been one of the main contributors. Personally, I’ve had the pleasure of working in the file format subgroup for the last 20 years, and had the honor of serving as vice chairman. The subgroup is still very active and published the sixth edition of the ISO base media file format in 2020.

File format structure

The primary purpose of the ISO base media file format is to carry synchronized timed-based media, such as audio and video, and enable efficient search and playback. This is achieved by a flexible object-oriented structure consisting of boxes labelled by size and type (function). Media bitstreams are stored in a media data box, whereas the logical structure of the file is stored separately as meta data in various functional boxes carrying details on tracks, dependencies, and timing information. The physical box structure is very flexible and easy to parse, which in turn makes the file format suitable not only for playback, but also for capture, editing and download.

Figure 1: Simple ISO file containing video and audio.

Following the demands of new applications, and taking advantage of novel features of new codecs and media types, the MPEG file format subgroup has continuously developed the file format by adding new tools and functionalities. Common structures that can benefit many use cases are preferably added to the media agnostic – and now Emmy® award winning – ISO base media file format, ISO/IEC 14496-12. Other structures that are suitable for certain media types or applications are defined in supporting standards of the ISO file format family. Examples of the latter include:

• Storage of AVC (H.264), HEVC (H.265) and VVC (H.266): Carriage of network abstraction layer (NAL) unit structured video in the ISO base media file format, ISO/IEC 14496-15
• Timed text and other visual overlays in ISO base media file format, ISO/IEC 14496-30
• Common encryption in ISO base media file format files, ISO/IEC 23001-7

Currently MPEG is also working on file format support for new media types including volumetric video and point clouds, as well as haptics for vibrations and other tactile inputs.

File formats and applications

Based on these structural specifications, there are many derived file formats for use in various systems and applications. MP4 is probably the most well-known example defined in MPEG. It fully exercises the general nature of the ISO base media file format and can carry MPEG-4 presentations and media streams, which provides a dynamic framework including many codecs and formats. A more recent example of a derived file format is the High Efficiency Image Format (HEIF), ISO/IEC 23008-12. It’s not intended for video per se, but rather image sequences and bursts compressed using HEVC.

Figure 2: The ISO base media file format forms the base for derived formats, such as MP4 , 3GP, and HEIF, and is also used by application formats, such DASH and OMAF.

3GPP was one of the first organizations outside MPEG to adopt a file format based on the ISO base media file format. 3GP files carry multimedia suitable for mobile devices, such as the 3GPP-defined speech and audio codecs AMR, AMR-WB, AMR-WB+, and EVS, as well as video codecs H.263, AVC/H.264, including stereoscopic 3D video, and HEVC/H.265. Timed text has also been supported from the first version of the format.

Popular examples of derived formats and their file extensions include:

• MP4 file format, ISO/IEC 14496-12 (*.m4v)
• 3GPP file format (3GP), 3GPP TS 26.244 (*.3gp)
• Motion JPEG 2000, ISO/IEC 15444-3 (*.mj2)

In addition to local playback of a file, there are many other applications of the ISO base media file format. Dynamic adaptive streaming over HTTP (DASH), standardized by MPEG (ISO/IEC 23009-1) and 3GPP (TS 26.247), uses segmented multimedia files for adaptive bitrate streaming over fixed and mobile networks. Another example is the Omnidirectional media format (OMAF, ISO/IEC 23090-2) for storage, delivery, and rendering of 360° video including, audio, images, and timed text. It uses the ISO base media file format for storage and access of immersive presentations for rendering on screens and head-mounted displays.

Compatibility

The ISO base media file format is exceptionally well suited for extensions, which has not only kept it up to date, but has also allowed features to be added while maintaining backward- and forward compatibility. A so-called file type box in the beginning of the file is used to signal compatibility through a list of brands. The major brand identifies which specification is the best use of the file, whereas any compatible brand identifies a specification it is compatible with. File format boxes that are not recognized by these brands can be ignored.

This way, an old file can be played by any player that recognizes its brand and it’s also possible to signal in a recent file that it can be played by an old player, conforming to an old specification, although only parts of the content may be rendered. For 3GP files, the brands are used to identify conformance to the different 3GPP releases and also to specify one or more profiles that signal whether a file is suitable for progressive download, adaptive streaming, or extended presentations with scene descriptions, and so on.   

Awards ceremony

The Emmy® Award for the ISO base media file format will be presented at the 72^nd Annual Technology & Engineering Emmy® Awards Ceremony on Sunday October 10, 2021 during the National Association of Broadcasters (NAB) Show.

Learn more

Read about another multimedia standard:

• VVC: The key to next-generation media services
• Versatile Video Coding explained – the future of video in a 5G world.
• Cutting the bitrate with Versatile Video Coding

Read more about 5G use cases:

• The emergence of the Internet of Skills
• Bring on the private networks with critical capabilities
• Read our report on gaming on the move and streaming video.