Skip navigation
Like what you’re reading?

Interpretability and fairness of machine learning models in telecommunications

  • As AI becomes integral to telecom, interpretable and fair ML models are crucial for building user trust, ensuring efficient network operations, and accurate resource allocation.
  • Regulators, including the European Union, are pushing for transparency and fairness in AI, with guidelines like the AI Act proposal to demystify and mitigate biases in AI applications.
  • In this blog post, we will explore interpretability and fairness methods in computer-vision-based ML models for telecommunication systems and demonstrate how these methods can diagnose and enhance model performance.

Senior Data Scientist with the Ericsson Global AI Accelerator

Data Science Intern

Interpretability and fairness of machine learning models in telecommunications

Senior Data Scientist with the Ericsson Global AI Accelerator

Data Science Intern

Senior Data Scientist with the Ericsson Global AI Accelerator

Contributor (+1)

Data Science Intern

As artificial intelligence (AI) continues to evolve and become an increasingly integral part of telecom products and services, interpretability, and fairness of the machine learning (ML) models used in telecommunications become more critical than ever. An interpretable ML model increases the user’s trust in the system by providing insights for model diagnostics and troubleshooting. On the other hand, a fair ML model, ensures unbiased decision-making, meaning it does not favor or discriminate against particular groups or categories. 

The negative effects of biased models are easy to notice in policies that affect people’s lives. However, bias can also cause problems in other models, like those used in telecommunications. In telecommunications, biased predictions can make networks run inefficiently or assign resources incorrectly, which can harm the system’s performance. A fair and unbiased model is essential for ethical considerations, legal compliance, social responsibility, and the overall effectiveness and credibility of the model.
In this blog post, we will delve into various interpretability and fairness methods implemented in computer-vision-based ML models in telecommunication systems. We will also demonstrate how these methods for interpretability and fairness can help diagnose and improve the performance of ML models.

The regulators, and particularly the European Union, have proposed or imposed measures to tackle this issue. For example, the AI Act proposal [1] lists practical guidelines for ethical AI use that align with AI principles from the Organization for Economic Cooperation and Development (OECD) [2]. These guidelines include transparency, which dictates that efforts should be directed towards demystifying AI applications, making their operations and decisions understandable and explainable to users and stakeholders. The guidelines also include fair application, which states that strategies should be implemented to detect and mitigate biases in AI applications, thereby promoting fairness and preventing discrimination. 

Ericsson is committed to implementing a trustworthy AI approach across its products, features, and services [3]. This approach incorporates the interpretability and fairness methods we explore in this blog post.  

A note about models 

Established and simpler statistical and ML models like linear regression and decision trees are generally easy to understand and interpret. As ML models, including those used in telecom products, become more complex and sophisticated, they often function like black boxes. This makes their predictions less interpretable and their biases or fairness harder to assess. 

An example of a complex ML model includes cutting-edge methods like transformers [4], which are now widely implemented in deep-learning, large language-, and computer vision models. 


Transformers are deep neural networks that can model long-range dependencies and leverage large-scale data for their training.  They rely on a mechanism called "attention"[4], which allows the model to focus on different parts of the input sequence when making predictions, helping it understand the context and relationships between the data more effectively. The transformer’s architecture consists of an encoder that processes the input and a decoder that generates the output. Layers of attention and feedforward neural networks are used to capture complex patterns in the data. This set helps identify how distant data elements influence and depend on one another.

In this post, we will use an object detection transformer (ODT) [5] as a use case to discuss interpretability and fairness. ODT identifies and locates objects within images by predicting bounding boxes and classifying them into specific categories.

 end-to-end object detection transformer

Figure 1: An end-to-end object detection transformer.

The end-to-end ODT uses a transformer architecture [4] to predict bounding boxes and class labels directly from images. It employs an attention mechanism to efficiently handle the relationship between objects and their contexts, streamlining the detection process. 

A complete analysis of interpretability and fairness methods is beyond the scope of this post. We present a real use case [6] by Ericsson using the ODT to transform a streetlight into a 5G site. Ericsson’s new street and outdoor small-cell solutions can be seamlessly integrated into existing infrastructure, such as streetlights. To achieve this, we developed the ODT model capable of detecting streetlights in images. This blog post explores how we leverage interpretability and fairness techniques to have more confidence in our solutions. 

Interpretability 

Interpretability can be defined as the degree to which a human can understand the cause of a decision [7]. It can also be defined as the ability to provide explanations in a manner that is understandable to humans [8].

An interpretability example that takes a picture of a cat

Figure 2[9]: An interpretability example that takes a picture of a cat (on the left) and plots a feature attribution map (on the right).

A feature attribution map is a visual representation of which parts of the image the model considers most important for its decision-making process. In this case, the grayscale image shows the attribution map, where darker regions correspond to areas of higher importance to the model. This interpretable image indicates that the model has learned to recognize key features that distinguish a cat, particularly around the cat's ears, eyes, and whiskers. 

Transformer interpretability 

We compare two classic methods and one state-of-the-art method for transformer interpretability in this blog post. The development of interpretability techniques for attention-based models has evolved significantly. Initially, the primary method involved extracting raw attention maps. This method visualizes direct attention weights produced by models like transformers, showing how much attention each input token gives to others within a layer. Raw attention maps can be helpful, but they have limitations. They only show token interactions in the last layer and miss the cumulative dependencies across multiple layers. As a result, they can be noisy and less informative for understanding complex transformer models.

To address this limitation, attention rollout [10] improves upon the raw attention map by assuming equal contributions from all attention heads and combining layers linearly. While it offers clearer insights, it still risks emphasizing irrelevant tokens and lacks class-specific interpretations.
 
Building on these advancements, generic attention-model explainability (GAE) [11] takes a step further by recognizing that different attention heads contribute variably, using a weighted sum for analysis. GAE provides a deeper understanding of model behavior than raw attention or rollout alone. Together, these methods reflect a progressive improvement in attention-based interpretability, thereby enhancing our ability to analyze and explain complex model predictions.
 
Despite significant progress in transformer models, achieving true interpretability remains a challenging task. Current methodologies exhibit several limitations such as skip connections may be oversimplified; the information flow between layers is typically treated as linear, and the relative importance of attention heads is primarily estimated through gradient approximations [12].

Transformer interpretability in ML models for telecommunication systems

Today, it's all about 5G experience and ubiquitous coverage, especially in urban areas and hotspots. Ericsson’s [6] new street and outdoor small-cell solutions are designed to be mounted on existing infrastructure, such as streetlights. By utilizing machine learning to detect streetlights from images, the need for manual intervention can be significantly reduced, speeding up deployment.  
However, for such a solution to be viable, the ML model must not only be highly accurate but also interpretable. This is critical because stakeholders, such as engineers and decision-makers, need to understand and trust the model’s decision-making process, ensuring the right infrastructure is identified consistently while avoiding costly errors. 
 
We used the three techniques (raw attention map, attention rollout, and GAE) on an object detection transformer trained to detect four types of poles on the street: streetlights, utility poles, hybrid poles, and straight poles (Figure 3). Streetlight poles illuminate roadways, while utility poles support power and communication lines. Hybrid poles combine light and utility functions for space efficiency, and straight poles serve a basic vertical design. Each pole consists of the pole structure and the head component, wherein the head component consists of the functionality of the pole. The straight pole does not have a head component. 

Different types of poles

Figure 3: Different types of poles

Reference image and three interpretability methods.

Figure 4: Reference image and three interpretability methods.

The figure depicts the reference picture (streetlight on the right and utility pole on the left). The ODT correctly identified both poles. The 6-part panel on the right shows the feature attribution maps from different interpretability methods. The top row is for the streetlight and the bottom is for the utility pole. The GAE method produces the most relevant visualization highlighting the most significant attention patterns. 
 
The three interpretability methods depicted address the classification of the four pole categories. In the raw attention maps (first column), all pixels of the streetlight and utility pole are highlighted, not just the crucial areas for differentiation. This approach includes the entire pole structure and head component, resulting in a noisy and less interpretable image. 

The attention rollout method (second column) remains somewhat noisy and is only partially effective in emphasizing the most significant connections associated with the pole. In contrast, the GAE method (rightmost column) focuses solely on key features, specifically the head component of the pole. These highlighted areas are crucial for determining the type of object identified, whether it be a streetlight, utility pole, hybrid pole, or straight pole. 

To further illustrate the effectiveness of the GAE method, we present another example below that exclusively highlights GAE's capabilities. 

An example of the GAE on a hybrid pole, a straight pole, and a streetlight.

Figure 5: An example of the GAE on a hybrid pole, a straight pole, and a streetlight.

The top row shows different poles highlighted in the image. The bottom row presents the corresponding GAE feature attribution maps, showcasing the key features identified for each pole from the image. 
In Figure 5, GAE is employed to analyze how the model classifies various objects into specific categories. For the hybrid pole and the streetlight, the model focuses on the heads of the poles, which aids in distinguishing between the two types of poles. For the straight pole, the model emphasizes the overall pole structure. 

In the above two examples, GAE methods provide more interpretable, noise-reduced, and comprehensive visualizations of attention mechanisms. They highlight key features, help filter less relevant attention weights, and aggregate information across layers and heads. 

Fairness in machine learning models 
 
The unfairness in these models can stem from three primary sources[13] 

  1. Data: Bias can originate from skewed datasets, unrepresentative distributions, and proxy attributes that misrepresent certain groups. 
  2. Algorithm: The design of the algorithm itself, including aspects like smoothing and regularization, can introduce biases. 
  3. User interaction: Bias can also be introduced through user decisions, such as ranking or prioritization, which may favor one group over another. 

Numerous methods have been proposed to assess unfairness in algorithmic models and systems. Among the most widely used approaches is the analysis of two key metrics: true positive rate (TPR, the proportion of actual positives correctly identified), and false positive rate (FPR, the proportion of actual negatives incorrectly classified as positive). Previous research [14][15][16] has demonstrated that by evaluating these metrics, we can effectively gauge unfairness in ML models. A fair model ideally produces comparable TPR and FPR across different demographic groups. 
 
In this blog post, we demonstrate through a straightforward and realistic example, how TPR and FPR can help us evaluate and mitigate bias in ML models through the same ODT model previously analyzed for interpretability.  
 
Fairness metrics of the ODT for Telecommunication systems: 
 
We use the same trained ODT model and measure the TPR and FPR for the different categories of poles.  It is assumed that the classification labels used to train the ODT also contain sensitive information. This assumption is relevant because if there are substantial differences in the performance metrics across different groups/categories, it indicates potential unfairness in the model's predictions. This understanding aligns with research indicating that ML models can unintentionally perpetuate biases present in training data, which may lead to unfair outcomes. 

Class 

TPR  FPR 
Straight poles  0.617  0.001 
Hybrid poles  0.878  0.009 
Streetlights  0.866  0.004 
Utility poles  0.829 0.015 

Figure 6: The table indicates different fairness metrics for the ODT 

The observed differences in TPR and FPR among different classes indicate a potential bias in the ODT model against utility poles. 
Specifically, utility poles exhibit a lower TPR and a higher FPR compared to hybrid poles and streetlights, suggesting that the model may struggle to accurately identify utility poles. We also observe that straight poles have a lower TPR compared to the other classes. 
 
Analysis of the training dataset revealed a potential bias stemming from the labeling of some small or partially obscured utility poles.  This may have been caused by human error during the labeling process. Additionally, the similarity in head components between utility poles and hybrid poles might contribute to the challenges the model encounters when distinguishing between the two. The lower TPR of the straight poles occurs due to the under-representation of the class in the training set. 
 
Various methods can be applied to reduce human labeling errors. Among the easiest ones, annotations can be validated by a second reviewer to improve accuracy. This process helps distinguish similar components, like the head structures of poles, and improves the quality of the labeling by reducing misclassification due to intra-class similarities. We have also used data augmentation to artificially increase the size of the utility poles and the straight poles, which can help the model generalize better and reduce inference errors. 
 
By applying these techniques, we retrained the model and recalculated the TPR and FPR metrics. The updated results indicate that the model is now less biased against utility poles than before. 

Class  TPR  FPR 
Straight poles  0.68  0.004 
Hybrid poles  0.865  0.009 
Streetlights  0.851  0.006 
Utility poles  0.843  0.012 

Figure 7: The table indicates different fairness metrics for the ODT after debugging for human errors and data augmentation. 

Conclusion 

In this blog post, we explored various interpretability methods applied to an ODT model in telecommunication systems. In the use case we presented, the GAE technique delivered the best performance by highlighting key features, filtering out less relevant attention weights, and effectively aggregating information across layers and heads. This approach provided valuable insights into the ODT model's decision-making process. 

Additionally, we discussed fairness in ML models, examining the sources of bias. We applied fairness metrics to the ODT model, using TPR and FPR to assess its performance. This analysis helped identify and address biases against utility and straight poles, ultimately improving the model's fairness. 

The Ericsson Blog

Like what you’re reading? Please sign up for email updates on your favorite topics.

Subscribe now

At the Ericsson Blog, we provide insight to make complex ideas on technology, innovation and business simple.