Bringing stories to life through immersive AR

It is an early morning in June. It’s still a bit chilly as you stand and look out over a plain from a vantage point close to the Dinosaur National Monument on the Colorado and Utah border. The sun has risen over the mountains and it looks like it will be a nice day. As you look out over the plain you see several brontosaurs walking towards you in the distance. Above them are a few pterosaurs flying in the sky. They soar over you and are flying so low, you can see these scary flying reptiles quite clearly – the details on their wings catch your attention. The sun is reflecting off their bodies and their shadows shoot pass you as they continue their journey over a ridge.

Turning your attention back to the plain, the brontosaurs have walked steadily closer to the spot where you’re standing. As they make their way past you some 50 meters away, a female stops to eat from a tree, and you can see how her heavy torso moves as she breathes. She whips her remarkably long, detailed tail, and for a moment you get the sense that she’s watching you.  

This could have been a scene from one of the Jurassic Park movies or from a book describing the long-gone life of dinosaurs in the Morrison Formation in the US. In this case, however, it’s a description of what augmented reality (AR) experiences will be possible to create and deliver in the very near future.

We are confident that in the next several years, you’ll be able to stand in the Morrison Formation (or other locations of your choice) and experience AR stories so realistic that you can barely distinguish what is real and what is not. To use my example, the level of detail is so real that you’ll be able to see which of the brontosaurs have scars from a fight with a T-Rex last year.  You’ll be able to see the sun reflecting off the dinosaurs and shadows being cast in the right direction.

What is augmented reality?

So, what is AR and why is it becoming so hyped up just now? Well, with AR technology, you can add digital assets to the real world. We could say that you’re able to enhance the real world or merge the digital with the real world.

In the past, you had to be in front of a 2D screen to consume digital content overlayed on the real world. Traditionally, the most common way to experience this would be an action movie with digital special effects. As technology evolved, we could add digital assets to real-time experiences in apps such as Pokémon Go. The more advanced the technology becomes, the more realistic the experience becomes and the blurrier the lines are between the real world and the digital world. This is true both from a timing and realism perspective. There is a convergence going on. But this convergence is not only happening between the real and digital worlds.

Let’s think ahead to where this evolution and convergence will take us. Eventually, we’ll be able to move from interacting with the digital world on 2D screens on smart phones and tablets to interacting in the converged real/digital world through head mounted devices (HMD). As Meta (previously Facebook) described at their rebranding event in 2021, the possibilities and opportunities are endless. This is also the reason why technology giants such as Meta, Google and Apple are investing massive amounts into AR/MR/VR. Think about it: the software and hardware platforms that we use today will evolve over time to become 3D AR/MR platforms on HMDs.

EPHAS: How we’re enabling today’s AR evolution

Over the last few years the EPHAS team and I with the support of Ericsson ONE, have  built an E2E augmented reality storytelling platform called EPHAS  – a location-bound platform that uses 5G, Edge and AR to share stories and events. It addresses the creative and consumption side of immersive AR storytelling while also democratizing AR storytelling.

Today, anyone who wants to create AR experiences will typically have to bring in a skilled developer to produce the experience and build the app through which it can be consumed. The problem is that if the storyteller wants to make any changes or add more stories in the future, the developer needs to come in again to make these changes. In essence, it is an expensive, un-scalable, custom-made solution.

In addition, the solution the developer made will become obsolete as technology evolves.To solve this problem, we’ve built a storyteller studio in which a layman storyteller can easily compile advanced AR experiences using any media of their choice. Think of the solution as a mix between a film editing tool and a game creation engine. The storytellers can then publish those AR stories that the consumer can access though an app on their own personal device. Today, those devices are typically smartphones and tablets. Tomorrow, they will be head mounted smart glasses. The stories are stored in the cloud and as technology evolves EPHAS is updated so the latest and greatest technology is always used.

Under the hood of an augmented reality platform

There are several more components needed to make super immersive AR stories come to life. First, you need extremely powerful computing capacity to manage many data-heavy detailed assets, how they interact with your device (how you’re moving and where you’re looking, for example) and how the assets interact with the surrounding environment (such as how light reflects and how shadows are displayed).

Second, when you have computed the experience on the edge or near cloud, you need to communicate the experience to your device in near real time. If you want to trick the human brain that the experience is real, you’re only allowed a few milliseconds of delay. A side effect of a delay is that the viewer can feel nauseous. This is where the low latency high bandwidth 5G solutions come into play. They contribute the important connectivity in immersive AR solutions by providing a fast-paced pipeline to transport the experience from the near cloud to the device.

So what needs to be done for this vision to happen? First, the technologies needed must be good enough to deliver these experiences. Secondly, the ecosystem of technologies needs to be connected and work in symbiosis. And thirdly, to achieve mass adoption, the cost for the end user needs to be low and the platforms needs to be easy to use.

On the first criteria, the technologies needed to deliver fully immersive interactive AR experiences are becoming a reality. AR technology is becoming readily available, the devices are becoming potent, development of HMDs are intensive, 5G and edge computing are being rolled out. The industry is working hard to connect the technologies into a well-functioning ecosystem, bringing down the cost and improving ease of use for end users. It might not be possible to say exactly when the mass adoption explosion will happen, but all the signs suggest that it will happen in the coming few years.

What is Ericsson’s role in the ecosystem described above and in the converged real and digital world? Throughout our 150-year history, Ericsson has created technology to connect human beings. A corner stone of our human connection is storytelling. With EPHAS and its new AR storytelling platform, we’re continuing our tradition to enable human communication and connection.

AR is one of the most tangible 5G use cases out there right now. By connecting the possibilities of 5G and edge computing with an easy-to-use AR story creation platform, imagine which stories and historical events you could immerse yourself in to. Imagine the things you could discover.

Want to learn more?

Read our blog post about 5G and augmented reality.

Read more about augmented reality games