- Drones, or unmanned aerial vehicles (UAVs), with video cameras are being increasingly used for tower inspections, reducing the number of tower climbs.
- New methods are being developed to improve video-based tower inspections using AI machine learning techniques.
- The UAV streams video to the ground operator’s mobile device. A computer vision application then processes the video frames to detect issues in real-time.
Reducing tower climbs
Radio towers are crucial to communications service providers’ radio access networks, as they elevate base station equipment connecting mobile devices to the network to improve cellular coverage.
When a problem with radio performance was traced to a specific radio tower, a service technician typically climbed the tower first to assess the situation, and then again to rectify identified issues and/or to upgrade the equipment.
Recently, the use of drones, or unmanned aerial vehicles (UAVs), for tower inspections for installations and maintenance has rapidly gained favor, reducing the number of tower climbs needed. UAVs equipped with video cameras have been the core of an emerging inspection ecosystem which also includes post-processing elements such as photogrammetry (taking measurements from photographs and video images) and 3D imagery.
Ericsson Research is developing methods to enhance and improve video-based tower inspections using AI. The UAV streams video to the ground operator’s mobile device, which then uses a computer vision application to process the video frames and detect issues with the radio tower installation in real time. The application can identify potential problems with coaxial feeder cable weatherproofing, color coding (to identify crossed cables) and cable bend radius measurements.
These issues can be diagnosed by utilizing the AI technique of deep learning – training a detector such as a convolutional neural network (CNN), with enough instances of “bad” versus “good” cabling installations to yield good detection results. However, unique characteristics of each case can require additional image-processing steps. In order to illustrate this, the case concerning cable weatherproofing is examined in detail overleaf.
Remote detection of coaxial cable connections for faulty weatherproofing
A common maintenance task with coaxial feeder cables in radio tower installations is identifying faulty weatherproofing clamshells. Plastic clamshells (consisting of a hard-plastic casing with soft gel interior) protect coaxial feeder cables between a radio unit and an antenna from rain and humidity by forming a seal around both the cable and connector. When a plastic clamshell fails to seal properly, an exposed connector may eventually impact data traffic passing through the cable.
One approach for detecting “loose” clamshells with a video camera-equipped UAV is by analyzing video images using a neural network that addresses a dataset of “bad” examples (see image A below). The detector, in this case, is coded to recognize an object or “class” in machine learning (ML) terms.
An open-source CNN framework was used to configure a custom neural network as a detector to identify faulty clamshell installations. The detector was designed with very lean computational requirements, enabling it to be run on a technician’s mobile device. The neural network consisted of eight convolutional layers performing feature recognition and six layers performing max-pooling operations for data consolidation.
Two practical issues affected the accuracy of this simple approach:
- First, the clamshell’s distance from the connector is not fixed, as it may move down the cable or stay higher, depending on the installation and issue at hand.
- Second, the connector is reflective, which creates a number of difficulties for the detector. It reflects background colors which differ from tower to tower and it also has specular reflections. On a sunny day, an exposed connector will reflect light with higher intensity than when it is cloudy resulting in the detector not recognizing it.
One solution is to account for dataset variability by providing more training data. However, this would require an extensive data-acquisition phase, with hundreds of thousands of samples, combining clamshells in different positions with varying lighting conditions. Such data gathering is time-consuming and expensive. Instead, additional refinements can be explored using the existing dataset.
An approach to solve the problem of variable distances between clamshell and connector is to train the neural network for two classes instead of one: a class for the clamshell and another for the connector. The detector can then identify these classes and localize them on video frames by drawing bounding boxes around them. Some conclusions can then be made by comparing their relative locations. For example, if the bounding boxes are aligned horizontally but their distance on the vertical axis is greater than a defined margin, we could deduce that the weatherproofing came loose (see image B below).
There is still the problem of properly detecting the connector due to its reflective metal surface. A solution was to pre-process some of the dataset, replacing specular reflections with gray as a placeholder. The same substitution was subsequently used on the UAV’s video frames, before executing the detector. In this way, the detector could match the connector on the video frame to the connectors it was trained with (see image C below).
Artificial intelligence (AI)
AI involves research into mimicking human cognition by developing algorithms for learning and reasoning. AI has existed since the beginning of digital computing, and has spawned an increasing number of sub-fields dedicated to developing techniques applicable to real-world problems in science and business.
Machine learning (ML)
ML is a major field within AI, involved in applying various methods of classification to large data sets and training models to recognize patterns. Uses include aiding computer vision applications.
Deep learning is a class of ML algorithms that use multiple layers of processing to extract patterns or features from data sets. Each layer uses output from the previous layer as input.
Convolutional neural networks (CNNs)
A powerful deep learning method is to employ a CNN as a detector in a computer vision task such as object recognition. Over the last decade, research into CNNs and fast implementations of CNN frameworks on high-performance graphics processing units (GPUs) have dramatically improved computer vision applications.
Benefits of removing specular highlight reflections
Precision (the rate of correct diagnosis) and recall (the ratio of objects detected out of total objects) are metrics used in assessing classifications and are standard measures of performance of an object detector in computer vision. The figure above uses these metrics to illustrate the benefits of adding an object class to the detector and then improving the results even more by removing specular highlight reflections from the dataset prior to training.
For precision close to 1 on the graph, all solutions behave the same, meaning all objects detected are actually the “real” objects (i.e. true positives). However, at the same time, the recall value is low and therefore many more undetected real objects exist. On the other side of the plot, however, solutions start to diverge. For higher recall values, whereby more objects are actually detected, removal of specularity yields higher precision, so more of the objects detected are true positives and not false ones.
Yet, even in the refined solution there is room for improvement. An even safer deduction of which clamshell belongs to which connector could be achieved by tracing the cable from the weatherproofing to the connector, using image processing techniques like thresholding to subtract background and edge detection to identify the cable. This illustrates an attribute of working with deep learning – one can increase performance by either enlarging the dataset or introducing additional steps into the process.
Deep learning is part of a bigger picture
Deep learning using CNNs should be seen as one powerful method in a growing arsenal of computer vision techniques which collaboratively support an array of use cases. For radio tower inspection, and more precisely, inspecting coaxial cables between radio units and antennas, deep learning is used in conjunction with specularity removal. Additional image processing techniques could include morphological transformations such as dilation and erosion, edge detection and contour tracing. The cabling issues described here are examples of the applicability of deep learning to mobile radio site management. It can be applied for classification and diagnosis wherever imaging is used for inspection.