The spectacular rise of holographic communication

What?

Ericsson Holographic Communication enables fully immersive, real-time, 3D experiences that can be built with consumer-grade mobile devices over 5G, while providing a greater sense of spatial awareness, proximity and presence.

Why?

Growing awareness of the benefits of holographic communication is driving rapid growth in its deployment across a wide range of sectors, from medical imaging and telepresence to remote collaboration and co-creativity.

How?

Improvements in artificial intelligence and new or enhanced capabilities will enable networks to adapt, depending on user behaviour and routines, while pre-empting what will be required from the network to enhance the overall quality and reliability of the experience.

Ali and Natalya, founders of Ericsson Holographic Communication, reflect on the possibilities ahead.

The takeaway

Holographic calling is the service that enterprises rate as the most important in improving their existing experience with remote communication.

Soon, every human sense will be digitally transformed, enabling fully immersive experiences that will change how we interact and communicate forever.

Holographic communication—a fully immersive, three-dimensional experience with spatial awareness—is available today, but there are barriers to greater adoption. It is extremely data-hungry, so new processing techniques are required to enable more mass-market solutions

By migrating processing to the network, we will soon be able to increase holographic field-of-view and resolution, while introducing highly intuitive levels of interaction.

Coupled with improvements in environmental and/or locational awareness, this will make it almost impossible to distinguish between virtual and physical reality.

Holographic Communication is no longer a thing of science fiction.

A composite illustration showing construction as an application for holographic communication

How many people now suffer from video call fatigue and want a more intuitive, collaborative and immersive way to communicate? To achieve an immediate, emotional connection, we must look beyond the existing experience of communicating with friends and colleagues on a flat, two-dimensional screen.

Holographic Communication represents a major leap forward in this regard, helping us all move to a fully immersive, three-dimensional experience with spatial awareness. In fact, in a recent Ericsson Consumer Lab report, holographic calling was the service that enterprises rated the most important to improve their existing experience with remote communication. To continue to attract and engage new talent, communication that allows for more authenticity and the sharing of non-verbal cues will increasingly be expected.

The success of any technology is, however, heavily geared around acceptance. How many new technological solutions have gained worldwide interest but then disappeared after a few years? To build on the sense of excitement from early adopters and the hype that follows, clarity on how Holographic Communication will evolve, how and when it will become available to a broader market and, crucially, why it’s better and more intuitive than alternatives, is needed.

Based on qualitative and quantitative market research, it is clear that for Holographic Communication to become an accepted form of communication it will require:

  • A flexible, scalable, and resilient platform to support the technology
  • Novel and socially acceptable XR devices (such as glasses)
  • A simple, attractive, and robust solution that is accessible and intuitive to all

When this does happen, Aaron Growsky, COO and founding member of Dreamscape Immersive, suggests that AR glasses "will become as personal to you as your phone or laptop". Peng Jin, Co-Founder of Nreal, agrees.

"[The glasses will] transport a digital world directly into the real world adding new layers of what could be experienced."

Pen Jin, Co-Founder of Nreal

Making it happen

Ericsson Holographic Communication is taking human communication to this next level: enabling immersive, real-time 3D experiences that can be built with consumer-grade mobile devices. With this holographic technology closely replicating face-to-face interaction, the work is opening up a wide variety of applications in areas such as manufacturing, utilities, and education.

Creating more of an emotional connection is vital in remote forms of communication, and Holographic Communication will go a long way towards achieving this. It will overcome that sense of distance with friends and family in other countries and help sales and marketing teams understand and empathize with their customers more. When this happens, the possible benefits to industry and society are vast.

As a technology, it maintains the three principal steps to bring an image to life, but differs from other similar solutions:

  1. Capture and pre-processing: The process of creating a measurable 3D representation of an object, person, or environment, and cleaned to remove any redundant points or noise to reduce the data size.
  2. Data compression and transmission: For real-time services, holographic communication will need to bring data rate requirements to be transported over wireless networks. The development of 3D compression algorithms is accelerating, and the efficiency of these algorithms will depend on hardware support on devices.
  3. Reconstruction and visualization: Decoding and rendering of the transferred 3D stream in real time, so that the experience can be recreated for a given viewpoint in the remote location.

Avatar technologies

That said, the output can look vastly different depending on the technique used and the quality of the image. In some solutions, avatar technologies based on artificial intelligence (AI) are used to generate a photorealistic representation of a user, to produce what are called ‘stereotypical holograms’ or synthetic avatars. These holograms provide some level of immersion, but in comparison with other options do not offer a high level of presence or a natural appearance of the user.

Camera-generated holograms

Another solution allows a near-cinematic experience, in which studio recordings are used for real-time capturing of holograms from multiple cameras to generate, as opposed to avatars, a real representation of users. These give high-definition equivalents of others and create the sense of immediacy between the users, due to the level of detail and interaction they provide. It does, though, involve a complex installation of a broad range of devices in a dedicated location, and therefore has limitations for rapid deployment and reuse. That means this approach is currently limited, as it is not scalable or accessible for most people.

Consumer-accessible holograms

With Ericsson Holographic Communication, the image is captured using consumer-grade phones or tablets, complemented by AI, to generate a full 3D holographic experience. This is made possible through leading-edge capabilities that compress video from 2 Gbps to 30 to 50 Mbps, which reduces the demand over the mobile network.

The compressed content is then transmitted to an off-the-shelf XR device (such as glasses), where the holographic stream is decoded and processed, before rendering in the user environment and displaying it accurately. The whole process takes a matter of milliseconds. The image is not a complete replica of the user but is a high-fidelity digital equivalent that creates a stronger sense of realism and emotional connection between the users.

Holographic communication is blending the real world and the virtual world into a cyber-physical continuum

The beginnings of Ericsson Holographic Communication

Intrapreneurs Ali El Essaili and Natalya Tyudina, who identified a need for more focused research into holographic communication at Ericsson and gained support for their ideas through the Ericsson ONE internal ideas accelerator, have spearheaded projects that have resulted in our technology running on 5G with off-the-shelf AR glasses. As of early 2022, the team is in talks with multiple leading operators around the globe to support in the development of a diverse set of consumer and enterprise use cases—and this is only the beginning.

“People look back and say, ‘I can’t believe people watched movies in black and white.’ We want to build something that will make people say, ‘I can’t believe people talked to other people over a flat video.”

Alvin Jude Hari Haran, Ericsson Holographic Communication team member

Holographic Communication in 2030

With millisecond latency, the time between a response from an action provided by 5G, the ability to distinguish between hard and soft objects (haptic abilities) and to mimic your movements and resistance through remote robots (kinaesthetic capabilities) are improving at an incredible rate. By 2030, this will be extended further so that every human sense will be digitally transformed to allow for fully immersive experiences. The ability to virtually teleport yourself to a remote location will blur what you can experience in both the virtual and physical worlds.

To contribute to a true sense of presence with Holographic Communication, environmental or location awareness is essential, too—and turning this into reality is something Ericsson is focused on.

How many times have we seen avatars walk through objects or hover above them? These glitches impact the overall experience, regardless of the fidelity of the image or the aesthetics of the XR device, causing a natural detachment from the level of engagement. If a hologram could understand its environment and react to it, it could theoretically recognize and move around objects more naturally. This level of real-time interaction would give a true sense of presence and connection to the physical world.

Hybrid group environments

Imagine visiting a café where there is a mix of physical and virtual discussions taking place, and avatars, people, and objects interact with each other seamlessly without glitches. Or perhaps a hybrid meeting in which everyone attends as a holographic image, occupying a seat within a physical room. Furthermore, presentations would no longer need to be two-dimensional, but would rather be included in the overall experiences as a further augmented overlay.

Haptics in virtual engagement

If a different perspective of a specific image or graphic was required, any attendee could rotate it in real time and leave annotations that could be referred to instantly or later in the meeting—a form of instant messaging during meetings but in a holographic/virtual form. Going one step further with tactile or haptic devices that allow a sense of touch in these environments, elements such as the texture, weight and elasticity of any physical object could be digitally transformed to be a part of a virtual world. This could create an ‘if you were there’ tour of a product in real time with others, offering an immediate impression of the look and feel, regardless of shape or size.

Shopping by hologram

From a retail perspective, holographic personalized assistance from any part of the world could be a possibility too. In a virtual store a hologram of a shop assistant could appear and a virtual replica of an item could be shared, allowing you to examine it both visually and by touch. Modifications to the item could be made in real time to create the perfect result and then be sent to you physically or shared as an augmented equivalent to use at future holographic/virtual events. Can you imagine what a virtual fancy-dress party would feel like? People from around the world could be there and, in principle, there would be no boundaries to how outrageous the costumes could be.

The technology that will make it work

To make it almost impossible to distinguish between a virtual and physical reality, a platform that allows ultra-low latency, high bandwidth, and intelligence to constantly adapt the availability of resources is necessary to combine and synchronize the two worlds. With 5G, huge steps have been made by going some way to establishing the ubiquitous connectivity and remote computing power that will be possible with 6G.

With the support of AI, further network capabilities will also be introduced to meet the demands of Holographic Communication. For example, a combination of the two will enable the network to constantly adapt depending on a user’s behavior and routines, pre-empting what is required from the network to enhance the overall quality and reliability of the experience.

Furthermore, combining simultaneous localization and mapping (SLAM) and deep learning, it will be possible for XR applications to not only localize and build geometric maps, but to instantly identify their surroundings which will allow new and more immersive ways of interacting with virtual objects.

Finally, to increase the field of view and resolution, enable interaction using natural gestures, and increase the battery life of XR devices, processing must be migrated to the network. XR devices (glasses, or even contact lenses) could therefore be much lighter in design and allow higher degrees of functionality. Peng Jin, Co-Founder of Nreal, believes that by 2030 the shape and size of XR devices will be “equivalent to normal sunglasses” in terms of the frame and lens thickness. But to make this happen, he says “will require a huge computing and network upgrade, without trade-offs in terms of cost and quality”.

Are you ready for the next step in communication?

Ericsson believes that communication is fundamentally human. If you pause to think about the pace of innovation and the infinite number of ways to contact someone in our daily life, our progress is astonishing. With limitless connectivity and Holographic Communication, what happens next will have the potential to improve lives and redefine businesses.

A vision of 2030

Contributors

Joerg Ewert - Strategic Product Manager

Joerg Ewert

Strategic Product Manager

Like what we do?