Introducing the Camera Uplink Benchmark (CUB) Project
As the world’s mobile networks adopt 5G, it has become increasingly common to use the cellular network as the uplink for video surveillance systems and other types of connected cameras. Optimizing video encoding and transmission for these systems can be complicated: what are the benefits of using one codec vs. another? What encoding parameters are best for my deployment? How does one objectively measure and compare video quality? To help answer these and other questions, Ericsson and AT&T have combined their expertise to develop a benchmarking toolkit for connected cameras. This project is known as CUB – the Camera Uplink Benchmark.
Overview and how to use it
CUB encodes video clips, sweeping through various bitrate and video quality related encoding parameters, scoring the resulting video quality using VMAF and recording the execution time. It saves the resulting quality and performance data for each encoding job into CSV files. CUB can ingest user-supplied video clips so that the results are relevant to their own deployments. CUB currently supports the x264 and x265 software codecs, along with H.264 / H.265 on Nvidia’s video encoding hardware (NVCodec) present in most of their GPUs, and Intel’s QuickSync encoding hardware present in most Intel CPUs. CUB does not currently support AMD’s VCN codec framework, but support can be added if requested. CUB also doesn’t do anything with on-camera encoders. Instead, it addresses a common case where one or more high quality camera feeds are being dynamically transcoded by a local PC, usually with a hardware codec, before being sent on the uplink cellular connection. Downloading and building all the video codec software can be complex. CUB’s install scripts handle the complex task of downloading, configuring, and compiling the necessary video tools, leveraging Docker containers to make the process simple and repeatable.
Intended Audience
Cellular networks now underpin business and public safety video, from security camera trailers to body cameras and UAVs. Maintaining sufficient video quality is essential, but choosing encoding parameters—especially target bitrate—is challenging: too low and the video is unusable; too high and it overloads the uplink, causing delays. Efficient use of uplink bandwidth is crucial to support multiple cameras while preserving quality and keeping latency low.
CUB is for system designers that need to navigate the complex tradeoffs between different video encoding solutions in terms of their run-time performance and measured video quality. This could be useful in deciding whether your CPU is powerful enough to handle your desired number of video feeds using just a software codec, or if you need to look at a hardware-based video encoding solution. CUB can be useful in estimating the total uplink bitrate required for the planned number of video feeds and their encoding parameters. This in turn could help the system designer determine whether the uplink throughput requirement would exceed the capacity of a regular best-effort cellular connection. If so, the system designer could use a connectivity API (e.g. through Vonage) to multiply the radio resources that the cellular network allocates to this application.
Most video encoding solutions can work over a range of different bitrates; higher bitrates generally preserving more picture quality, and lower bitrates presenting more compression artifacts. Because the uplink bitrate capacity is not unlimited, it is important to get a rough idea of the range of bitrates required to preserve sufficient video quality without exceeding the capacity of the radio uplink. This can help determine how many cameras can share the same link and sensible video encoder target bitrate settings.
Opensource Source Code and Videos
CUB is available on github here: EricssonResearch/camera-uplink-5G-benchmark. It includes a README file to get started.
Additionally, we have video datasets to evaluate compression and analytics as samples using CUB. The dataset is distributed on the Consumer Digital Video Library (CDVL), by the Institute for Telecommunication Sciences (ITS) and hosted by NTIA (National Telecommunication and Information Administration).
Under the hood
- Uses BASH scripts to compile ffmpeg, VMAF, multiple software codecs, and related tools directly from source.
- Provides an automated script to select and encode video clips and capture execution performance data.
- Performs all compilation inside a Docker container (Ubuntu 24.04) to minimize compatibility and environment issues.
- Treats codecs, encoding presets, reporting, QoE metrics, and analyses as plug-ins, making it easy to add or swap components in the workflow.
Next Steps for CUB
- Test camera built-in codecs, e.g., via ONVIF
- Analyze radio uplink signal quality, rate adaptation behavior, and related APIs
- Add additional video quality metrics for more comprehensive evaluation (P.1204-series models, temporal features, …)
- Retrain VMAF for specific, targeted use cases
- Provide tools to generate plots and graphs for clearer visual analysis
Anticipated Impact: Advancing 5G Video Applications
This joint effort between Ericsson and AT&T is a practical initiative supporting next-gen 5G video transmission. It helps system designers and connected camera vendors to evaluate their current and future video delivery systems and tailor them to their use cases. Through this partnership, the benchmarking tools and methods offer:
- A basis to select encoding techniques for video uplinks supported by objective data
- A path away from one-size-fits all toward solutions adapted to each use case
- A common base framework for adding new metrics, sample videos, and analyses, where these improvements can be shared with others in the connected camera community
If you are interested in this space for optimizing video quality for connected cameras, feel free to reach out! We also welcome software contributions and feature requests.
Please reach out to us through the Discussions tab in the GitHub repository mentioned above
Read more
