Real-time water quality monitoring with data pipelines

With millions of devices being deployed across an increasing variety of locations and with a wider range of sensor inputs, this brings plenty of interesting possibilities for future sustainable societies. Real-time water quality management is an incredibly important aspect for any sustainable society and needs to be continuously monitored and improved, and any quality issues should be flagged at the earliest possible stage. At Ericsson Research, we are driving the evolution towards a sustainable and intelligent society by applying Internet of Things (IoT) solutions enabled with artificial intelligence (AI) capabilities.

In this blogpost, we look at how intelligent IoT solutions can be employed for real-time water quality monitoring and how, through important metrics and data, they can provide automated updates as to water quality for each intended recipient.

Why do we need real-time water quality monitoring?

The global availability of water resources has never been as scarce as it is today. At the same time, pollution levels in the water are imposing a bigger challenge than ever. Water is often becoming polluted without awareness; often due to the complex water distribution systems, where water is flowing in and out of the pipes.

Automated and real-time water quality monitoring solutions can provide timely information about water quality by directly processing the data collected from distributed monitoring mechanisms, thereby enabling quick responses to address potential leakages and water pollution incidents.

The challenges of monitoring water quality in real time

Real-time monitoring relies on the collection of sensor data, including low-quality raw data. This brings additional challenges when it comes to understanding and monitoring water quality. On one hand, measurement of water quality requires continuous monitoring over longer periods of time. On the other hand, in many cases the data describing the status or quality of the water has various restrictions when it comes to access privileges. Therefore, there is a need to be able to provide information to different parties, with different privileges and over extended periods of time.

In many regions in the world, raw data sets related to water quality cannot be obtained directly, mainly due to various regulations and data protection laws. In these cases, the required information is extracted manually which leads to a slow and cumbersome process. Thus, automated solutions are required to provide information on demand, without (or with as little as possible) human interference, to read and analyze the data and to provide insights into the data based on the privilege of the receiver.

Intelligence-enabled IoT offers a way to address problems such as these. This is done by establishing an end-to-end “data pipeline” to extract knowledge from raw data. It is a way to manage raw data and conduct knowledge in levels of availability to comply with data regulations.

What is intelligence-enabled IoT?

Intelligence-enabled IoT introduces intelligent functionality to the IoT ecosystem, onboarding AI capabilities to compose flexible end-to-end systems. Intelligence can be distributed either to the very IoT edge, running the analysis next to the data sources; or by composing chained analysis computations and data processing steps running anywhere from the IoT edge to the cloud.

Such distributed IoT systems with integrated intelligence capabilities can be used to form knowledge pipelines, extracting insights and various quality indicators from distributed data sources in different locations. These knowledge pipelines receive raw data and deliver intelligence in different levels of availability for users as the final outcome. The information can be categorized into three different layers: the raw data, the features of the raw data extracted using machine learning, and the final knowledge and insights about the data.

How can intelligence-enabled IoT contribute to real-time water monitoring?

By deploying such knowledge pipelines, we can address water quality monitoring in an automated way. By utilizing distributed and automated analysis, data can be made available for processing without relying on human analysis or operations. Thus, insights can be provided even without manually accessing the data.

Provisioning data at different levels of availability allows the data to be served against a variety of cases (see Figure 1 for a depiction of the availability levels). Some users require access to the raw data directly, other users need to access refined data with full details, whereas yet another set of users only require the final conclusion of the analysis. The design of such a knowledge pipeline can address all the needs of the different users.

Moreover, the knowledge pipeline provides a process to refine data, such that the raw data can be hidden and replaced by the extracted features or even only the final insights as the data passes through the knowledge pipeline. For example, in many cases when analysis is performed by humans, direct access to the raw data is not allowed due to various regulations. At the same time, the status of water quality or pollution levels should be available to the general public. With the restricted regulations, many data users are not able to access the sensor data collected by conventional IoT solutions, contradicting the public exposure of the information regarding the water quality and pollution. This problem can be solved by the automated knowledge pipeline, which automatically conducts analysis from the raw data and provides the information regarding water quality to the public, without depending on or interfering with the raw data analysis as the conventional data analyzing process does.

Figure 1: The automated knowledge pipeline formed by intelligent IoT systems for knowledge retrieval

The intelligence-enabled IoT pipeline for iWater

Water monitoring networks – iWater is one of the joint research projects funded by Vinnova where we together with our partners explore intelligence-enabled solutions for future sustainable smart living environments. Participants in this work include Stockholm municipality, Telia, KTH Royal Institute of Technology, Stockholm University, Linköping University, and Stockholm Water and Waste.

For the iWater project, the pipeline as described above is deployed beside Lövon lake, Stockholm, for retrieving insights from the water quality data in a digitalized way. In this scenario, we have sensors deployed in the lake near a water treatment plant, continuously measuring various characteristics of the water. The data is automatically collected and ingested into the data pipeline for instant analysis and backup. As part of the pipeline, various machine learning (ML) algorithms are deployed to analyze the raw data to refine and draw insights from the measurements regarding the water quality. The ML algorithms conduct online analysis on multi-dimensional time series data. This serves both the purpose of quality analysis of past data and early warning mechanisms to predict potential problems in the water quality.

Finally, the insights from the data analysis are also stored for long-term accessibility, and both the raw data and the refined data are published to a visualization platform, see Figure 2, where users can access the data without the need to request additional analysis or management. The privileges of the various data sets can be fine-tuned to provide more control over who can see what; for example restricting the raw water quality measurements to parties involved in research, drinking water facilities or smart city development, while refined information can be publicly available to help public health awareness.

Figure 2(a-f): A view of the iWater visualization dashboard, showing some of the sensor data analysis insights from the last 24 hours.

Read more

Learn more about the iWater project with Ericsson and Vinnova.

Take a look at the various biomimicry engineering concepts on display at the 2020 Ericsson Innovation Awards.