Metadata: a must for the digital transition of wastewater treatment plants

Abstract

[EN] The increment in the number and diversity of available (and affordable) sensors together with the advances in information and communications technologies have made it possible to routinely measure and collect large amounts of data at wastewater treatment plants (WWTPs). This enormous amount of available data has boosted the interest in applying sound data-driven solutions to improve the current normal daily operation of these facilities. However, to have a real impact in current operation practices, useful information from the massive amount of data available should be extracted and turned into actionable knowledge. Machine learning (ML) techniques can search into large amounts of data to reveal patterns that a priori are not evident. ML can be applied to develop high-performance algorithms useful for different tasks such as pattern recognition, anomaly detection, clustering, visualization, classification, and regression. These ML algorithms are very good for data interpolation, but its extrapolation capabilities are low. Hence, the data available for training these data-driven models require points covering the complete space for the independent variables. A significant amount of data is required for this purpose, but data of good quality. To transform big data into smart data, giving value to the massive amount of data collected, it is of paramount importance to guarantee data quality to avoid “garbage in – garbage out”. The reliability of on-line measurements is a hard challenge in the wastewater sector. Wastewater is a harsh environment and poses a significant challenge to achieve sensor accuracy, precision, and responsiveness during long-term use. Despite the huge amount of data that currently being recorded at WWTPs, in many cases nothing is yet being done with them (resulting in data graveyards). Moreover, the use of the data collected is indeed very limited due to the lack of documentation of the data generation process and the lack of data quality assessment. Metadata is descriptive information of the collected data, such as the original purpose, the data-generating devices, the quality, and the context. Metadata is needed to clearly identify the data that should be used for the development of data-driven models. These data should be selected from the same category. If we include data that shouldn’t be in the same data set because they were obtained under different operational conditions, this would lead to unreliable model predictions. ML algorithms learn from data, thus to be useful tools and to really improve the decision-making process in WWTP operation and control, representative, reliable, annotated and high-quality data are needed. Effective digitalization requires the cultivation of good meta-data management practices. Unfortunately, there are no wastewater-specific guidelines available to the production, selection, prioritization, and management of meta-data. To address this challenge, the IWA Task Group on Meta-Data Collection