Ericsson worldwide

Mixed reality and dialogues about future urban plans and developments

During the last years, we have together with both internal and external partners explored various aspects of ICT and sustainable urbanism. The ambition with the activities described here are also to continue to develop concepts and strategies within this broader theme, and to concretize what kind of values that new technologies might bring. It is still very much research-in-progress, so please see this blog post as notes from the first iteration.

The scope of these activities emerged in collaboration with Arup Digital Studio and UN-Habitat Urban Planning and Design Lab, with many of the discussions revolving around ways for urban technologies to engage and inform a range of actors, stimulate sharing of experiences, and help identify alternative paths of action. We did speak a lot about new types of development models that are light, quick and adaptable. And how ICT could enable a kind of public digital commons to facilitate different types of dialogues and interactions.

With 60 percent of the area expected to be urban by 2030 not yet built, urban planning is often highlighted as a key area for urban transformation. Contemporary thinking describe a change from masterplans to community visions, and that collaborations between a range of actors are crucial to capture emerging ideas and enable innovation. These new approaches require the involvement of the public in all phases, and that planning processes can respond creatively to the expressed needs of a city’s inhabitants.

While many cities have the ambition to engage citizens in early dialogues and co-creation regarding urban developments, they often lack the tools and internal structures to successfully maintain a continuous dialogue or doing it at a larger scale. As one of the tools, digital technologies are increasingly being explored for communication purposes and citizen dialogues. Among interesting examples are the use of 3D models, games, and virtual reality to make ideas less indefinite and reduce the uncertainty for all stakeholders.


3D model of Trondheim created with aerial and street photogrammetry

These dimensions where further investigated in an in-depth analysis of five aspects that are central to foster sustainable urbanisation. For a summary of that parallel and very much integrated track, including a social impact assessment of the concept examples below, please have a look at the previous blog post from our colleague Fanny Von Heland at Ericsson Research Sustainability.

Some ideas and concepts within participatory planning

As hinted in our recap of a previous mining project, we did already during that work begin to think about how detailed 3D models and mixed reality could have a value in an urban context as well. And in our conversations with both Arup and UN-Habitat, the need for visualizations of alternative developments to engage communities and facilitate better discussions was emphasized. Hence, we set out to further explore how a technology like mixed reality potentially could transform dialogues about future urban developments.

Architects and planners have for a long time utilised digital tools and models in their work, although not always making the information available in a legible format to the public. The ideas to somehow bridge the physical and digital worlds has also been around for a while, but the technology has not been quite there yet. Now we (and many others) can see concrete opportunities to demystify urban planning and make processes more transparent. The ability to experience alternative developments in real life – from locations, perspectives and times that matter – can make the process more engaging, increase legibility of information, and lower barriers for participation. Something that can help finding better ways forward based on people’s’ needs and desires, but also increase understanding for different perspectives.

The main track of conceptualizing aim to illustrate how mixed reality might enable citizen dialogues as an integrated part of people’s everyday situations, and thereby contribute a more inclusive form of digital public space. The examples range from low-fi to hi-fi, and from personal devices like a mobile phone or Google Cardboard style goggles, to shared virtual reality goggles available at the library, or public digital binoculars at city squares, bus stops, viewpoints, etc. The physical and visual characteristics of future built environments are important attributes, but the broader approach include a range of ways to experience (actual and simulated) environmental information and facilitate access to urban data.


Images adapted from illustration by AIX for zoning plan developed by Stockholm City and Besqab

City-scale mixed reality applications

While urban planning and citizen participation are recognized as key areas for future cities, and served as a starting point for our concept development, a city-scale mixed reality platform could certainly be valuable for a wide variety of processes and tasks. Transportation systems, navigation, gaming, signage and information, architecture, geodesign, spatial analysis, property management, emergency services, art, and cultural heritage to mention some areas.

It is intriguing to think about how many different objects could benefit from having augmented superpowers, and based on a spatial awareness being able to respond through various modalities of mixed reality (think a mobility aid with haptic feedback, or an umbrella that displays certain environmental data inside the canopy). So, zooming out from the mixed reality + participatory planning context we are looking at what kind of key technical functionality that could enable a city-scale mixed reality platform to support a range of applications.

Current state-of-the-art augmented reality systems rely on visual mapping and tracking. One limitation is that these technologies typically require a mapping procedure to be carried out before graphics can be rendered and overlaid. Another limitation is the coverage of physical space due to limited reach of sensors. Hence, experts in the field point toward the need for commonly accessible 3D mapping data of (all) physical environments in order to enable instant, mobile and real-time tracking and positioning. This is something that would be crucial for mobile city-scale mixed reality applications and devices.

On that note, city-wide 3D mapping projects like the one that is being carried out in Stockholm, Sweden presents very interesting opportunities. 4000 km of streets, bicycle lanes, and walkways have been laser scanned and photographed. The initial purpose is to measure and monitor the urban environment, but this data might have a great value for many other applications – mixed reality and dialogues about future developments being one category.


Image by Stockholm City Department of Traffic showing merged 3D point cloud and photo

In order to enable mixed reality experiences on a range of cheaper, lighter, lower power, smaller devices (like actual glasses), the precise positioning must be possible to do with simple local sensors and with low energy consumption. The need for significant amount of processing power and storage space is another limiting factor for non-premium devices. This means that less on-device processing is desirable, which in turn puts high requirements on low latency and reliable connectivity.

As one activity to investigate high performance cloud components and innovative media technologies for mixed reality content delivery, we have begun to explore how 3D point clouds can be dynamically streamed based on location and viewing direction. And how algorithms for occlusion and level of detail can be taken into account to effectively manage large (city-wide) point clouds, which may serve as the commonly accessible 3D mapping data of the physical environment.


Point cloud from Stockholm City Department of Traffic visualised in a streaming compressor by Umbra

Research in progress – more to come

We will continue to develop our thinking around key assets for mixed reality in cities, and the technology components that enable 3D mapping data of urban environments. As part of this, we aim to test our hypothesis out in the reality and will, for example, engage with partners to further concretise how mixed reality (in combination with other technologies) might be strategically integrated with urban planning, citizen participation and collaborative governance.

Look out for updates about these activities on the blog. In the meantime, do not hesitate to contact us with comments, questions, ideas, etc.

Written by Marcus Nyberg

Marcus explores and conceptualises how new technologies might enable innovate ways to address societal challenges.

Commenting rules