Mixed Reality Mining – ICTs in a mining context
During last spring we at Strategic Design and the Service System Research team engaged in a joint research project with Boliden. Together we explored what the mining industry could do with 5G and ICTs in a near future. Our goal was to develop tangible concepts showcasing how Ericsson’s current or possible future technologies could create benefits for the mining industry; in terms of safety, productivity, employee well-being, or any other relevant aspect. We strived to take the mine worker’s perspective and not come up with concepts increasing productivity but disregarding the miners’ experiences. In the end, the outcome turned out to not only be relevant for mining, but also interesting in a range of other contexts.
Understanding the context
We kicked-off the project by visiting Boliden's gold mine in Kankberg in the north of Sweden. All of us from Ericsson considered a mine visit as necessary, as no one had any prior experience of neither what it is like being approximately 500 meters underground, nor the mining processes. We did not really know what to expect on beforehand, but it became evident that pitch dark is really very dark and that it is not at all easy to find ones way around in a mine if you do not know its layout inside out. Also, that working as a miner can be very lonely. For example when working long shifts at a rock face. And that meeting huge trucks down there in the narrow tunnels can be pretty scary (now they have drivers in this mine, but imagine when some of those are self-driving). Another insight was that quite a lot of time is "lost" every shift transporting people up and down, and to safe locations during blasts.
The project team at the first visit in the Kankberg mine
Automation + humans
Much of the advanced technology development in the mining industry is currently revolving around mine automation and autonomous vehicles. But with introduction of automation, human workers are often disregarded. And it will probably take a long time, if it ever will happen, before all mining operations can take place without any (or with minimal) human involvement. Therefore we decided to not primarily focus on autonomous mining, but on the (semi-)manual skill based work and the people involved. That is, how to assist and improve the work by means of technology, and the interplay between automation and humans.
We had no lack of interesting ideas during our ideation workshops. Remote operations enabled by 5G and fine-grained positioning were topics that were recognised as important early on. Our main challenge was to narrow down the scope and decide what to focus on. But after grouping, sorting and having turned ideas inside out, we could identify two bigger themes connecting a lot of the ideas. These were two (interconnected) platforms allowing for a range of use cases and applications.
The virtual mine model
The first (and major) track is based on a 3D-model of the mine and includes data from all its activities, updated in close to realtime. This virtual replica can be generated from lidar scans and sensor data, and be used for remote operations, traffic control, automation, productivity aids, visual alterations, safety measures, and so on. It could for example allow for "time travelling" by exploring a "tomography" of a rock face over time. We used this platform to illustrate a range of virtual, mixed and augmented reality applications, which could be used both for operations down in the mine and remotely from above ground.
Traffic control and geologist support experienced with VR in the mine model
The social web of the mine
The second area can be described as an IoT platform for the mine (conceptually similar to SWOT) focused on contextual, targeted and relevant communication and information, and potentially building on the same mine model. The purpose of such system would be to reduce manual administration and allow for better production planning, facilitate communication between mine workers, to build social cohesion, and increase the understanding of tasks and their relation to other activities in the mine, filter information, turn invisible data into useful information, to mention a few (well, that was maybe more than just a few).
We thought of using the radio communication as the primary interface for the system, as the miners already are in the habit of using audio/verbal communication. But an alternative, or complementary, interface could of course be graphical. That is, some sort of news feed or visual notifications which can be arranged depending on object, action, timeline, responsibility, or similar.
So why is this interesting? For Boliden, the benefits in terms of safety, productivity and employee well-being are clear. For Ericsson, the scope extends far beyond the mining context, making it even more relevant. We foresee that equivalent technologies and principles can be applied in other contexts, but the mine is naturally a very contained and controlled environment suitable for the first iterations of explorations.
Thinking of the forthcoming fleet of self-driving cars and LIDAR scans of cities that are being done, those could for example provide a city with more detailed data about its streetscape almost in real time. A platform made up of LIDAR data combined with existing static 3D models (such as from aerial photos or CAD-files) generating an accurate and continuously updated 3D model, could be the base for a wide range of AR/MR/VR applications in cities, just like in the mining context. In the broader city context this could be valuable for processes and tasks like simulations, prototyping, visualisations, evaluations, inspections, remote control, etc. City planning and urban development would, for example, be a very interesting application area allowing for more understandable simulations and enabling co-creation, democratising the process. Maybe even creating a new kind of digital public space.
We will post more updates on cities and urban development ahead, over and out!