Welcome to a world of immersive experiences with XR

These are exciting times, as it always is when you’re at the threshold of a new paradigm. It will soon be time to look up from our phones, straighten our backs and step right into the digital experiences that have been caught behind our screens for so long. We’ll be able to invite the digital realm into our real world and share it with others. Welcome to the immersive era of XR.

Time to look up from your phones: XR devices will release us from our downwards gaze on a small screen and will take us to where our gaze naturally wants to go.

So what is XR? Let’s rip off the band-aid at once. The term XR means ‘eXtended Reality’, meaning technology that lets you experience and interact beyond your physical space and the reality of what you can see, hear or feel. You’ve probably heard about augmented reality, mixed, merged and virtual reality – all terms which describe different technologies that add to or change our experience of reality.

While the academic definitions and scope of these terms are much discussed, XR can be used as the umbrella term for all of them. Having our everyday lives, our entertainment, and our work empowered and enhanced by XR technology has been anticipated for years. And for years it has been evolving.

XR: The next paradigm

So why can’t we just continue using our smartphones to connect with each other, play games, or shop? Could we not just develop augmented reality apps that could summon cute monsters, new furniture inspiration or a hologram of your colleague through your phone? Although most smartphones are ready to provide you with AR experiences, the smartphone user interface has its limitations. For example, your arms tire from holding a phone out in front of you, and the interaction would still be limited to the screen.

Think about it. Today, many of us look into a small rectangular screen for hours a day – even when we’re out and about. For 15 years, we’ve been helped and enchanted by the mobile internet and our smartphones. Everything you can think of is at arm’s length, right in the palm of your hand. It’s great and its convenient. But it has also left us with our necks bent unnaturally to see that tiny screen. It has interrupted countless conversations so that we can check the directions, and who knows the number of real-life interactions we’ve missed as our eyes are fixed on endlessly scrolling through our feeds. We miss out on the context and the people around us. XR-based devices will allow us to bring our head up and bring information to where our gaze naturally falls or where it is contextually relevant to see it. Interfaces will move from being screen based to voice or gesture based. With augmented reality, we might even see the development of new graphical interfaces used for writing or silent navigation for example, where they could appear just as if they were a real keyboard.

Extended reality devices are expected to bring immersive experiences that merge digital objects, information and overlays into your everyday reality.

Like deep fake videos that fool the eye and ear with lifelike copies, which can be both entertaining and scary, the blend of digital and reality can also give us fantastic and unrealistic additions to our daily lives. The point is that digital or remote visuals, such as rendered objects or holographic video with spatial sound and even the possibility to touch, will be integrated with our physical context of reality.

Together with spatial sound and haptics (technology that can create the experience of touch), the digital and physical worlds will become increasingly indistinguishable from each other.

Welcome to the world of the Internet of Senses.

Extended reality applications

In our research at Ericsson, we have seen significantly increased interest in XR applications this last year, compared to 2019.

While the gaming and entertainment industry has been the driving force of XR applications and device development for many years, pandemic restrictions have pushed the market readiness and demand for remote working, learning, shopping and socializing in more immersive ways than ever before. Of course, we’ve all missed the excitement of being part of a crowd dancing to the beat, hanging out with friends in restaurants, and (even if we might not have expected it) looking our colleagues in the eye in a cramped conference room. But the past year and a half has also taught us that some activities, without doubt, can be done remotely and digitally – we’ve likely just missed the feeling of presence, the emotions and the finer details.

XR is all about giving you access to those details and turning that flat screen into an immersive and meaningful experience. For work meetings and classes, this could be a holographic representation of colleagues or students, or spatial sound that places you in a group of people where discussions can flow naturally. But XR can also take advantage of spatial understanding, object detection and the mapping needed to make applications work – for example, providing smarter and more customized solutions for consumers.

Gaming and entertainment

Let’s start with the obvious application area. For years, gaming has been driving XR experience development, both in terms of devices and applications. While VR gaming could be considered a niche market, new opportunities promised by augmented reality glasses are seeing more people look forward to immersive gaming – in fact, two thirds of consumers want to try augmented reality gaming and even a third of people who don’t game today . With discreet, lightweight glasses that blend the digital game into your real environment, it’s easy to imagine how gaming will attract new audiences in new environments and make it more of a social and shared experience.

Already today gaming encompasses everything from complex virtual multi-worlds to mobile puzzle games. With XR technology we’ll possibly see a range of interactive entertainment and gamified educational applications too. Elsewhere, fitness culture has already taken tech and biometrics to its heart, and cyclists who can afford it can buy AR glasses that display their speed and distance as they bike. In our gaming research, 43 percent of respondents claimed to be very interested in AR sports. I won’t be surprised if we’ll soon be able to compete with a projected avatar of our best selves when going for a jog, or work out with a virtual, eternally enthusiastic personal trainer supervising our performance by our side.

Read more about XR over 5G

Learn how time-critical communication capabilities in 5G networks will enable breakthroughs in a wide range of application areas, including extended reality (XR).

Click here

Work life and education

Remote work and telepresence have become an increasingly interesting area for a broader audience over the last year, for obvious reasons. Working from home (WFH) has no doubt proven its potential and shown us various benefits when it comes to cost and sustainability. But the flaws of today’s remote presence tools have increased our expectations and demands towards more immersive features.

During the pandemic, my colleagues and I used VR to get around the worst of interacting in regular video meetings. For all the times video meetings fell flat, VR creative workshops, intense discussions and social gatherings were great. Not only did we meet at a paradise island with the sound of gentle waves, but we also appeared as good looking avatars with groomed haircuts which clearly did not reflect the reality of our real world lock-down looks. We could actually look each other (through our avatar) in the eye. But the best part though was the natural flow of conversation with gestures and no cross talk.

We’re also expecting a change over the coming years in enterprise structures and organizations that will drive the implementation of XR applications even further. The ‘De-materialised office’ and ‘Future of enterprises’ reports have showed the rise of interest in XR among white collar workers. We’re already seeing the opportunities of working from anywhere and employing people from anywhere. This includes the beginnings of new migration patterns – for example, educated people moving out of expensive cities to get more space – and change that is driving shifts in company culture and employee expectations. The question is whether there will be a need for investing in offices at all in the future, or if they will transform into fancy conference facilities, instead of work cubicles to suit new ways of meeting and working.

The idea of immersive presence by remote persons, or transporting yourself to be immersed in a remote place, has a broader relevance than office work. In manufacturing, so called fully automated factories could be visited via XR by implementing a digital twin into the factory, which makes it possible to walk close to active robots. Telepresence also has immense potential to bring value and access for all in the fields of education and healthcare. Not to mention how nice it would be to chat with friends and family in a holographic call instead of over a flat screen with bad audio.

Retail and shopping

Fashion is not what most XR enthusiasts talk about as the future of tech adoption, perhaps because you don’t need to care about your real looks if you are in VR. Jokes aside, retail was one of the sectors that changed most in the last online revolution. XR applications have huge potential to change the paradigm of sales, shopping and services, since they could fill the gaps that today’s online retail applications cannot deliver, such as inspiration, customisation and ‘try before you buy’. 

It’s even possible that it’ll be in retail that we’ll see the fastest adoption of haptics and other multi-sensory digital experiences as part of audio-visual experiences - once the ecosystem of sensors and devices have developed for the mass market. What if you could virtually try on a jumper or a shirt, and visualize it fitting perfectly to your lifelike avatar copy who acts as your mirror. Would you not also want to feel the material, to judge if it’s cotton, silk or wool to know if it’s what you want?

45 percent of early adopters say that they want all senses virtual shopping malls.

How will the retail landscape change in a future where inspiration and ‘try before you buy’ trials can be experienced as part of your home as holographic renderings? Will your everyday expenses increase or decrease? Will the problem of overconsumption become even more heightened when you can purchase a pair of shoes just by looking at the cool ones the person in your local café is wearing?

Today’s online shopping, algorithms and advertising makes many of us buy too much.  But if designed with purpose, XR and AI could help us shop less with better results. Knowing the quality and perfect fit of products could lead to fewer returns – and fewer unused and unnecessary products piling up in our homes.

One of the more exciting questions is how we’ll shop for fashion and design if digital objects could be perceived just as well as real ones? Already today you can buy digital items from luxury brands to use with your gaming avatar, and there are fashion designers who sell digital-only items that you can wear on social media without filling up your wardrobe.

On the note of social media, we could wonder how face-mounted cameras will change the way we take pictures for social media. Will the entry of mass market AR glasses also mark the exit of the selfie (which already uses AR-powered augmentations to lift your cheeks, smoothen your wrinkles, widen your butt). Will we return to turning the cameras towards our surroundings instead of ourselves? Will the AR-fuelled XR era turn the pendulum from egocentrism and help us focus more on others? Well, to be honest I think XR could just as well bring us the next level of the selfie: Placing personal avatars in front of the camera instead of yourself. As humans, we’ll always behave in certain ways. We’ll always have dreams and aspirations – but how we achieve them changes over time depending on the technology, products and services available as society changes.

The future of extended reality and the possibilities

And just as human behavior is a constant, so are the worries and threats. The dark side of digitalization will follow us whether in smartphones or AR glasses, and we must be aware of the risks. What happens when ransomware moves into XR? What does identity theft mean in XR, and how can you be protected from being replicated as a hologram that asks your grandmother to send money to someone else’s account? Another question is what attitudes we will have towards always-on cameras on AR glasses everywhere. Will other people get angry because you might be filming them or even worse, volumetrically capturing them, i.e., making a 3D recording of them? And how will companies and governments regulate the use of such devices in offices, labs and police stations?

There’s no doubt, we need to explore the risks as soon as possible, and develop smart ways to avoid the problems that might arise if we want to benefit from all the great uses of XR. Because it’s not the technology that should be steering us – we humans need to carefully design what the technology should enable and what we want to do with it.

To me, the greatest promise of XR is the potential of doing more with less. The possibilities that arise can lessen the need for travel and commuting. Enabling digitalized products show the potential to cut carbon emissions and the over-use of natural resources through the de-materialization of consumption.

I also believe we will not only consume XR applications. We will be part of creating the next world of XR. When inviting friends for dinner, you could place colorful birds chirping around a lamp. At Christmas, the neighbors could have reindeers grazing in their snowy garden, and during the holidays, the kids could spray paint the living room with their art - all digitally of course. Even more thrilling is the prospect of designing your own digital outfits to wear in the view of anyone wearing AR glasses (if you don’t, it’s probably best to keep your boring old physical clothes on underneath).

Finally, I look forward to a world without screens as a filter, that lets us step into truly immersive entertainment without bulky headsets. A world without tedious video meetings where the most common sentences are, “Can you hear me? Can you see my screen?”, where instead we meet eye to eye with colleagues who are joining in a way that best suits them wherever they see best. A world of applications that are designed to help us when we want to be helped, and won’t disturb us when the real world needs our attention. A world of digital access that does not strain our necks and cripple our thumbs, but puts the interface where it will be used: on our eyes, ears and skin, so that we can behave like the humans we are.

Learn more

Find out here: Are we living in a simulation?

Are you ready to experience new dimensions of reality?

Read our blog post How will 5G and edge computing transform the future of mobile gaming?

Read our blog post AI bias and human rights: Why ethical AI matters.