Co-creating the city — City-scale mixed reality tested in Johannesburg
We just returned from Johannesburg, South Africa, where we tested an early prototype of a mixed reality platform that we are working on. The experiment in Johannesburg was part of a collaboration with UN-Habitat, Johannesburg Development Agency (JDA) and Wits University where we are jointly exploring digital innovation in community-driven urban design.
UN Habitat, who we have been collaborating with for the last three years (read more here and here), have been running a program called block-by-block since 2012, were they arrange workshops in urban communities around the world, using Minecraft as a tool that people who are neither computer-savvy nor knowledgeable in architecture or city planning can use to draft their own proposals for things that could improve their city.
Together with UN Habitat we arranged a pilot workshop during the Fakugesi festival in Johannesburg where our particular focus was to explore how mixed reality could work as an additional component in the block-by-block workshops.
The site for this pilot workshop in Johannesburg, a small public place in Braamfontein in downtown Johannesburg, was chosen together with JDA and Wits after visiting a number of locations in the city that were prioritized areas for remodelling and general improvement. The venue was a street-level art gallery facing the place at the intersection of Stiemens and Bertha street. We ran workshops during three days with festival goers and passersby dropping in as well as invited students from neighbouring Wits University and Rosebank College.
Each workshop session started with a brief introduction to Minecraft (if needed) and the purpose (make something that improve the adjacent public place), after which the participants discussed and created proposals in a minecraft-replica of the area that we had installed on a number of computers. After finishing we copied their creations to our mixed reality application and they went outside and viewed what they had just made in Minecraft in real life using special smartphones enabled with sensors and 3D-sensing technology.
Video of workshop participants and some of their proposals, which they built in Minecraft and exported to our Mixed Reality prototype viewer. Real objects like buildings, signs, trees, streetlights etc are appearing either behind or in front of the virtual 3D objects, as it should if the digital bridge was physically there — a key characteristics of 'mixed' or 'merged' reality.
For the technically inclined, the devices we used for this test were Tango-enabled smartphones/phablets that has parts of the on-board SLAM functionality disabled, instead using our own pre-loaded 3D mesh based on a Lidar-scanned point cloud which we used for occlusion-masking, ray-tracing shadows etc.
Workshop participants exploring their own digital creations on-site in Johannesburg.
After the open workshops we collaborated with urban design postgraduate student, Klaas Tswai, at Wits University, who developed some of the proposals into a slightly more refined plan for remodelling the whole place. This refined draft proposal could also be explored on-site, in mixed reality.
A big 'thank you!' to all participants and partners who helped us arrange the workshop.
We are continuing to explore city-scale mixed reality and how it could be used in the facilitation of citizen dialogues and co-creation related to urban design and planning.
All photos by Joakim Formo/Ericsson Strategic Design