Creating 3D digital twins for indoor radio simulation

5G (and 6G) use higher frequency bands, including mmWave, C-Band, and upper mid-band, compared to previous generations of mobile networks. These higher frequencies offer increased network capacity but are more sensitive to obstacles and surface materials. Therefore, indoor radio access network (RAN) planning requires highly precise 3D modeling and simulation of indoor environments to address the increased propagation losses and sensitivity to obstructions.

To address this issue, since 2019, Ericsson has been exploring various approaches such as radio ray tracing, which leverages 3D gaming technology to enhance radio frequency propagation models and RAN deployment planning.  This initiative is part of a broader effort to adopt end-to-end digital twins, including our market leading Ericsson Site Digital Twin. We began with network digital twin (also known as, “subscriber digital twin”) and cell site twin (also called, “industrial digital twin”).

The main challenge in applying radio ray tracing is the availability of 3D indoor models of existing venues. While commercial 3D exterior models of buildings are available for outdoor venues in major urban areas, there is no equivalent for indoor venues in the current market. Increasing adoption of computer aided design (CAD) models and building information modeling (BIM) for new buildings is promising but they don’t address the needs of existing buildings, which mainly have paper blueprints and 2D floor plans in PDF format.

What technical options can we consider for digitizing existing buildings? Terrestrial laser scanner (TLS) offers excellent spatial resolution but is hard to scale due to cost and throughput. Some solutions using 360 cameras provide good visual representation, known as the “doll house view”, suitable for a virtual tour but not ideal for physical RF simulations due to numerous slits, holes, and irrelevant objects that significantly distort radio propagation simulation results. Also, neither of them effectively provides material information, which is critical for accurate radio simulation.

Simulation-friendly 3D structure extraction

To apply radio ray tracing to indoor venues, Ericsson is looking into ways to develop simulation-friendly digital twins of indoor venues that meet the specified requirements. For scalability across various venues, ranging from small offices to factories, a generic smartphone (iPhone15 Pro) is used for recording videos of a target venue instead of special devices like the TLS which cost thousands in USD for each.  Inertial sensors in the modern smartphone help estimate camera extrinsic in indoor environments, which is challenging for a typical structure-from-motion (SfM) or multi-view stereo (MVS) due to insufficient texture and frequent occlusion. As the base 3D representation, a point cloud is generated from the captured video frames for a given venue and located in a self-generated map, following a typical modern 3D modeling process for indoors.

A major challenge with existing buildings is that when we capture videos for modeling, the space is often already in use. This causes significant occlusion and clutter from furniture, partitions, and other items irrelevant to the static structure (such as walls, ceiling, and floor) which are crucial for simulation. To resolve the issue, a 3D object detection model is applied to the point cloud to differentiate between dynamic objects (like chairs and tables) and static structures. Once all the points are segmented by the object class, we can “defurnish” the room by removing unwanted dynamic objects. This makes it easier to extract static structures, giving the impression that the videos were recorded when the building was newly constructed and empty. Another benefit of this static-structure-only modeling is that it allows radio designers to test various scenarios in the same venue by placing mesh models of chairs, tables, partitions, and racks in the structure with an arbitrary layout.

 

Figure 1. Workflow of an indoor digital twin

 

Another key requirement, material prediction, is implemented at this stage using 2D material estimation models and the calculated relation between 2D images and 3D point clouds. By projecting material estimation results of multiple 2D images onto the shared 3D objects, we can set material properties to each point and subsequently to its derived mesh.

 

Figure 2. Examples of 2D material prediction using a fine-tuned deep neural network.

 

The last challenge is unique to radio ray tracing. Typical 3D mesh models which are generated from point clouds have lots of holes and slits. Although such disconnections appear acceptable to the human eye when visually rendered at room scale, they lead to radio signals leaking outward which significantly distorts RF simulation results. In short, to make your 3D room model compatible with the simulation, your mesh model should be watertight. To resolve the issue, we devised an algorithm for detecting semantic planes from the raw meshes and merging planes while keeping the global structure and thereby removing slits and holes between them.

Figure 3 below is the combined result of a meeting room in our Santa Clara office, followed by Figure 4 showing a real radio ray tracing result with the resulting model. For readability, we removed the ceiling and added text annotation onto the semantically color-coded planes of the watertight mesh. All the indoor models are saved in OpenUSD format for easy visualization and integration with raytracing models.

 

Figure 3. The material-tagged result from the indoor digital twin pipeline

 

Figure 4. Sample result of compatibility test with radio ray tracing simulation

 

Conclusion

A fully automated 3D indoor modeling pipeline is introduced to generate semantically simplified material-aware indoor mesh models supporting radio ray tracing simulation from a handheld camera. The telecom-friendly solution is expected to help scale up the new way of designing indoor radio networks using radio ray tracing simulation by accelerating its application to indoor sites.

Acknowledgment

A special thanks to the Indoor Digital Twin project members, Jianfu Zhang and Qing Wang, and to Magnus Lundevall for his invaluable feedback as a subject matter expert.