Time-to-content: Benchmarking network performance
Service providers are aware that a good user experience is key for driving customer satisfaction, thereby reducing churn and improving financial return. Here we outline an objective method to measure and benchmark the network performance required for a positive user experience.
Video streaming accounts for the majority of mobile network traffic, and has naturally been a focus area for improvement. Compression techniques and adaptive mechanisms have been very effective at lowering the time-to-play and adapting to changing network conditions. But while streaming is buffered, web services place higher demands on network performance as they require instant speed. Therefore, meeting the requirements for web browsing will also satisfy the demands of many other use cases, including HD video streaming and social media.
Web services are interactive, which means that every time a user makes a choice or sends a request by clicking on a link or filling in a form it sets up a delay that directly contributes to user experience. This is different to services like video, where users tend to stay with the content for a longer period. The responsiveness of the service is measured in seconds, and while a good time-to-content (TTC) is very much subject to context and user expectations, there are scales that can be applied, such as the one defined by Google in their Core Web Vitals developer framework.1
Web browsing versus video streaming
Webpages consist of many objects that are fetched to form a complete page. The design and type of webpages can have a substantial impact on TTC as shown in Figure 32. Moreover, web services are usually interactive with content leading to more user requests – each with its own TTC delay. This is unlike video streaming behavior, which is characterized by a request followed by a length of video played out. The interactivity and TTC for web services are what demand instant speed.
For video, the user experience is affected by both the time-to-play and avoiding stalls during play-out. Video streaming uses buffering to smooth out throughput variability.
Video is also a key element in a growing set of media that includes VR, AR and gaming. These are real-time services that generally cannot be buffered, as with most current streaming video services, so will increasingly put demands on network performance over time beyond those for web-browsing and related services. Current consensus is that these real-time applications will not be a material portion of mobile traffic before 2025.
Ericsson’s Smartphone Lab study
A selection of 50 popular websites were tested using an automated lab environment. Figure 32 represents the TTC measurement for the webpages at different downlink throughput rates. The results were calculated for each throughput in a range from 1–30Mbps. The results indicated a strong relationship between available downlink throughput and the TTC.
The results indicated a decreasing TTC (y-axis) as the downlink throughput (x-axis) increased. A regression model of the downlink throughput was chosen to fit a curve that would be representative of the results. Factors used in the regression analysis, including device performance, network latency and web content size, were then projected in a model for 2025 (Figure 33).
Some sites keep their webpage content intentionally low in order to be fast-loading with a reduced dependency on factors such as throughput. However, the majority have richer content with a high dependency. By taking the 75th percentile instead of the median, throughput requirements can be determined that are representative of a large number of websites.
With a TTC scale, which grades sites on a scale of excellent (<1.5s), good (1.5–2.5s), fair (2.5–4.0s) or poor (>4.0s), we can derive the throughput that would be required to meet a certain TTC target. For example, based on the results of the 2025 model, a TTC target of below 1.5s would require a throughput of at least 20Mbps (Figure 33).
What does this mean for service providers?
Service providers can adopt this approach while optimizing their networks. Throughput measurements taken on live network data sessions, and logged on performance counters in network nodes, can be benchmarked and plotted on a scale. This will help to identify and prioritize network improvements on the cell level in order to make the biggest impact on user experience possible for any given level of network investment.
A snapshot of the results from the Ericsson Smartphone Lab study
Figure 32: Automated lab environment website test results
Figure 33: TTC targets for 2025 model
1 Web Vitals, Essential metrics for a healthy site, web.dev/vitals.