Network performance through customers’ eyes

Key findings

• The survey found the two most important factors to customer satisfaction were webpage download time and uninterrupted video streams.
• Of those who are satisfied with their video streaming experience, 8 out of 10 are also satisfied with the mobile network experience.
• Network improvements resulted in an 11 percentage point increase in willingness to refer the mobile network to others.
• Understanding the relationship between customer satisfaction, experience and network performance is the key to designing networks to drive loyalty.

These insights were used to set up a nationwide measurement program of service key performance indicators (S-KPIs), ¹ which strongly correlate to customer satisfaction and can be optimized through root cause analysis of both radio access network (RAN) and non-RAN issues.

In 2017, Swisscom upgraded its mobile network in central Zürich by expanding capacity and introducing evolved network architecture. The objective was to improve customer experience despite increasingly demanding apps and growing network traffic. The project also aimed to prepare the network for a national rollout of gigabit speeds ahead of the introduction of 5G.

Over a three-month period, the mobile network underwent several improvements, including an evolution to Centralized RAN (C-RAN) architecture and the introduction of software to coordinate inter-site carrier aggregation.

When carriers are aggregated within and across macro and small cells, the probability of achieving performance gains increases. This leads to higher data speeds and results in a better customer experience. The solution also increases network efficiency, lowering the cost per gigabyte delivered. Through this approach, the network can be effectively coordinated to ensure the highest possible performance.

As Swisscom implemented the mobile network performance enhancements in a limited test area, the upgrade provided an opportunity to conduct the satisfaction survey with those who spend a significant amount of time each month in the same area. The objective was to understand the impact of improved network performance on customer experience, and included insights into:

• Usage of apps and services on the 4G network – respondents’ self-evaluation, validated by network data, can be seen in the figure above
• Issues and customer experience before and after network improvements
• How customers evaluate the performance of the mobile network
• Factors that drive network satisfaction and willingness to refer Swisscom to others

Improved minimum speed drives customer satisfaction

Before the network improvements were made, customers’ satisfaction with speed was lowest for data-intensive apps, especially when uploading or downloading photos and videos, or streaming music and video. When the survey results were compared after the network enhancement, Swisscom’s assumption that an improvement in minimum speed would result in a better customer experience was validated.

After the network enhancement, analysis of independent active speed tests on User Equipment (UE) in the test area showed a 33 percent increase in minimum speed to 8.5Mbps.

The customer survey data showed that a better experience was reported by 42 percent of daily streamers and 24 percent of all other participants. This indicates that heavy data users benefited from the network improvements. These improvements resulted in an 11 percentage point increase in willingness to refer the mobile network to others, while also reducing detractors by 26 percent.

Customers’ most important criteria when assessing network performance
After network enhancement, respondents’ satisfaction with coverage, reliability and speed had improved.

When asked how they evaluate the performance of the network, the top five criteria for all customers in order of importance were:

1. Time taken to open a webpage
2. How long it takes to transfer photos and videos in chat apps quickly e-mail attachments download
3. How long it takes for a streaming video to start
4. How often streaming video freezes

However, daily video streamers consider video start time and freezing to be more important than e-mail attachment download speed for evaluating network performance.

Speed as a quantifiable property was not important to most respondents. Only 10 percent reported that the results from speed test apps were their most important criterion for evaluating network performance. Nearly three-quarters of all respondents were not even aware of the maximum speed offered by their unlimited data subscriptions.²

In contrast, customers who were already dissatisfied with the mobile network ranked their criterion differently. For them, the most important indicator of network performance was video freezing, followed by results from a speed test app. A possible explanation is that this sample of users is young and digitally advanced, and therefore high-performing video is essential to satisfaction.
Improvements made to the mobile network reduced the occurrence of many frustrating issues often faced by respondents (see the figure above). Occurrence of the most frustrating issue – an unavailable network – was reduced by almost half. The second most frustrating issue – slow webpages – only improved slightly, whereas customers reported a strong reduction in the third most frustrating issue – video freezing.

As part of the second wave of the survey, Swisscom asked detailed questions about the video streaming experience after improvements were made to the network. Several aspects of the video experience were ranked differently in terms of their satisfaction and relative importance. Although customers were most satisfied with the quality of the video, the two most relatively important aspects for video satisfaction were smoothness of playback (no video freezing) and the time taken to load and start a video (see the figure on the following page).

The results also show that satisfaction with video streaming is strongly correlated with overall satisfaction. Of those who are satisfied with their video streaming experience, 8 in 10 are also satisfied with the mobile network experience. Further to this, 72 percent of users who are dissatisfied with video streaming are not satisfied with the mobile network experience. Those who are satisfied with video streaming are far more willing to refer Swisscom’s mobile network than those who are not. Of the customers who were satisfied with video streaming, 88 percent had no intention of changing operator within the next 12 months. However, only 35 percent of users dissatisfied with video said they would definitely not switch.

These results show that video streaming experience is critical to customer satisfaction and loyalty, especially for younger users, and that end-to-end network design and optimization should set the goal to improve this experience.

These results make it clear that not all aspects of the customer experience can be improved by enhancing the mobile network. Enhancing RAN does not completely address all customer experience problems. It is important to identify bottlenecks and optimize performance with an end-to-end perspective, including the app itself, the customer’s device and the path data takes through the internet. In the case of slow webpage download, improving response time on the internet requires an efficient Content Delivery Network (CDN) or troubleshooting of the Domain Name System (DNS).

The customer study explored in this article identified key performance indicators (KPIs) that have a strong impact on customer satisfaction. The two most important S-KPIs were:

• Webpage download time – the most important customer criterion for evaluation of the network
• Video freezing – a criterion that has a very strong influence on satisfaction with the mobile network

These S-KPIs enable the corresponding network performance KPIs to be identified, measured and optimized, ultimately increasing customer satisfaction and retention.

Swisscom has taken further actions based on these results, and is measuring these and other S-KPIs using passive probes. This gives a network-wide view of their status.

The S-KPI approach reveals the performance of apps and services, not only within single cells but also as customers move between cells. This detects problems that are missed when monitoring individual cells. The S-KPIs are correlated with data about the RAN in a time-stamped, end-to-end session record for each customer. The network is then optimized to improve the poorly performing S-KPIs.

The insights from this survey strongly indicate that mobile networks should be designed and optimized based on KPIs that accurately reflect customers’ perception of network performance and that are important for customer satisfaction and loyalty. The findings present a way to communicate with customers about the tangible benefits the mobile network delivers and which customers can directly experience, such as rapid webpage downloads and uninterrupted video streams.

The customer study and resulting insights described in this article are an important step toward understanding the relationship between customer satisfaction, customer experience and network performance. Swisscom began by investigating the impact of network improvements on customer experience and satisfaction. Once these relationships were established, the problem could be turned around; it is becoming possible to start with customer experience, then measure relevant S-KPIs and optimize the network to maximize satisfaction and loyalty. This approach will be further explored and tested to fully understand its potential.

Research methodology and segmentation

The survey was conducted in collaboration with Ericsson Consumer & IndustryLab, with 8,000 Swisscom customers invited to take part. Of these customers, 750 agreed to participate, all of whom spent at least 40 hours per month in the test area, with three-quarters living or working there. The others frequent this popular area to shop, meet with friends or clients and eat at restaurants, for example.

The survey took place in two waves: before any improvements were made to the network and one month afterwards. The 150 respondents who participated in both waves enabled a direct comparison of the customer experience before and after network enhancement.

The sample was evenly divided between genders and predominantly consisted of people from younger age groups. Only 17 percent were aged over 50 years old and 45 percent were under 30 years old. Most respondents (83 percent) were customers classified by Swisscom as digitally advanced users.³ Almost all respondents had subscriptions with unlimited data, although subscriptions differed in terms of maximum speed (largely between 10Mbps and 50Mbps).

Footnotes

¹ Service key performance indicators (S-KPIs) measure the perceived user experience at application level from an end-to-end perspective (from application servers to User Equipment)

² Survey respondents had post-paid subscriptions, with unlimited data volume and maximum speed limits depending on the subscription (for example: maximum 2Mbps, maximum 10Mbps or maximum 50Mbps)

³ Swisscom segments residential users according to usage of digital consumer devices, including smartphones, tablets, television and computers. The term “digitally advanced user” refers to customers in those segments that exhibit a strong interest in new technologies and use them fluidly in daily life