The spectacular rise of holographic communication

How many people now suffer from video call fatigue and want a more intuitive, collaborative and immersive way to communicate? To achieve an immediate, emotional connection, we must look beyond the existing experience of communicating with friends and colleagues on a flat, two-dimensional screen.

Holographic Communication represents a major leap forward in this regard, helping us all move to a fully immersive, three-dimensional experience with spatial awareness. In fact, in a recent Ericsson Consumer Lab report, holographic calling was the service that enterprises rated the most important to improve their existing experience with remote communication. To continue to attract and engage new talent, communication that allows for more authenticity and the sharing of non-verbal cues will increasingly be expected.

The success of any technology is, however, heavily geared around acceptance. How many new technological solutions have gained worldwide interest but then disappeared after a few years? To build on the sense of excitement from early adopters and the hype that follows, clarity on how Holographic Communication will evolve, how and when it will become available to a broader market and, crucially, why it’s better and more intuitive than alternatives, is needed.

Based on qualitative and quantitative market research, it is clear that for Holographic Communication to become an accepted form of communication it will require:

When this does happen, Aaron Growsky, COO and founding member of Dreamscape Immersive, suggests that AR glasses "will become as personal to you as your phone or laptop". Peng Jin, Co-Founder of Nreal, agrees.

Ericsson Holographic Communication is taking human communication to this next level: enabling immersive, real-time 3D experiences that can be built with consumer-grade mobile devices. With this holographic technology closely replicating face-to-face interaction, the work is opening up a wide variety of applications in areas such as manufacturing, utilities, and education.

Creating more of an emotional connection is vital in remote forms of communication, and Holographic Communication will go a long way towards achieving this. It will overcome that sense of distance with friends and family in other countries and help sales and marketing teams understand and empathize with their customers more. When this happens, the possible benefits to industry and society are vast.

As a technology, it maintains the three principal steps to bring an image to life, but differs from other similar solutions:

That said, the output can look vastly different depending on the technique used and the quality of the image. In some solutions, avatar technologies based on artificial intelligence (AI) are used to generate a photorealistic representation of a user, to produce what are called ‘stereotypical holograms’ or synthetic avatars. These holograms provide some level of immersion, but in comparison with other options do not offer a high level of presence or a natural appearance of the user.

Another solution allows a near-cinematic experience, in which studio recordings are used for real-time capturing of holograms from multiple cameras to generate, as opposed to avatars, a real representation of users. These give high-definition equivalents of others and create the sense of immediacy between the users, due to the level of detail and interaction they provide. It does, though, involve a complex installation of a broad range of devices in a dedicated location, and therefore has limitations for rapid deployment and reuse. That means this approach is currently limited, as it is not scalable or accessible for most people.

With Ericsson Holographic Communication, the image is captured using consumer-grade phones or tablets, complemented by AI, to generate a full 3D holographic experience. This is made possible through leading-edge capabilities that compress video from 2 Gbps to 30 to 50 Mbps, which reduces the demand over the mobile network.

The compressed content is then transmitted to an off-the-shelf XR device (such as glasses), where the holographic stream is decoded and processed, before rendering in the user environment and displaying it accurately. The whole process takes a matter of milliseconds. The image is not a complete replica of the user but is a high-fidelity digital equivalent that creates a stronger sense of realism and emotional connection between the users.

Intrapreneurs Ali El Essaili and Natalya Tyudina, who identified a need for more focused research into holographic communication at Ericsson and gained support for their ideas through the Ericsson ONE internal ideas accelerator, have spearheaded projects that have resulted in our technology running on 5G with off-the-shelf AR glasses. As of early 2022, the team is in talks with multiple leading operators around the globe to support in the development of a diverse set of consumer and enterprise use cases—and this is only the beginning.

With millisecond latency, the time between a response from an action provided by 5G, the ability to distinguish between hard and soft objects (haptic abilities) and to mimic your movements and resistance through remote robots (kinaesthetic capabilities) are improving at an incredible rate. By 2030, this will be extended further so that every human sense will be digitally transformed to allow for fully immersive experiences. The ability to virtually teleport yourself to a remote location will blur what you can experience in both the virtual and physical worlds.

To contribute to a true sense of presence with Holographic Communication, environmental or location awareness is essential, too—and turning this into reality is something Ericsson is focused on.

How many times have we seen avatars walk through objects or hover above them? These glitches impact the overall experience, regardless of the fidelity of the image or the aesthetics of the XR device, causing a natural detachment from the level of engagement. If a hologram could understand its environment and react to it, it could theoretically recognize and move around objects more naturally. This level of real-time interaction would give a true sense of presence and connection to the physical world.

Imagine visiting a café where there is a mix of physical and virtual discussions taking place, and avatars, people, and objects interact with each other seamlessly without glitches. Or perhaps a hybrid meeting in which everyone attends as a holographic image, occupying a seat within a physical room. Furthermore, presentations would no longer need to be two-dimensional, but would rather be included in the overall experiences as a further augmented overlay.

If a different perspective of a specific image or graphic was required, any attendee could rotate it in real time and leave annotations that could be referred to instantly or later in the meeting—a form of instant messaging during meetings but in a holographic/virtual form. Going one step further with tactile or haptic devices that allow a sense of touch in these environments, elements such as the texture, weight and elasticity of any physical object could be digitally transformed to be a part of a virtual world. This could create an ‘if you were there’ tour of a product in real time with others, offering an immediate impression of the look and feel, regardless of shape or size.

From a retail perspective, holographic personalized assistance from any part of the world could be a possibility too. In a virtual store a hologram of a shop assistant could appear and a virtual replica of an item could be shared, allowing you to examine it both visually and by touch. Modifications to the item could be made in real time to create the perfect result and then be sent to you physically or shared as an augmented equivalent to use at future holographic/virtual events. Can you imagine what a virtual fancy-dress party would feel like? People from around the world could be there and, in principle, there would be no boundaries to how outrageous the costumes could be.

To make it almost impossible to distinguish between a virtual and physical reality, a platform that allows ultra-low latency, high bandwidth, and intelligence to constantly adapt the availability of resources is necessary to combine and synchronize the two worlds. With 5G, huge steps have been made by going some way to establishing the ubiquitous connectivity and remote computing power that will be possible with 6G.

With the support of AI, further network capabilities will also be introduced to meet the demands of Holographic Communication. For example, a combination of the two will enable the network to constantly adapt depending on a user’s behavior and routines, pre-empting what is required from the network to enhance the overall quality and reliability of the experience.

Furthermore, combining simultaneous localization and mapping (SLAM) and deep learning, it will be possible for XR applications to not only localize and build geometric maps, but to instantly identify their surroundings which will allow new and more immersive ways of interacting with virtual objects.

Finally, to increase the field of view and resolution, enable interaction using natural gestures, and increase the battery life of XR devices, processing must be migrated to the network. XR devices (glasses, or even contact lenses) could therefore be much lighter in design and allow higher degrees of functionality. Peng Jin, Co-Founder of Nreal, believes that by 2030 the shape and size of XR devices will be “equivalent to normal sunglasses” in terms of the frame and lens thickness. But to make this happen, he says “will require a huge computing and network upgrade, without trade-offs in terms of cost and quality”.