Monitoring urban black-odorous water by using hyperspectral data and machine learning

Highlights

• Hyperspectral bands provide unique info concerning severity of black-odorous water.

• Factorial data mining synthesizes water quality parameters into reduced dimensions.

• Reduced dimensions establish correlation between water quality and spectral data.

• Machine learning based feature extraction selects critical spectral bands.

• Reduced dimensions and critical spectral bands reveal condition of water quality.

Abstract

Economic development, population growth, industrialization, and urbanization dramatically increase urban water quality deterioration, and thereby endanger human life and health. However, there are not many efficient methods and techniques to monitor urban black and odorous water (BOW) pollution. Our research aims at identifying primary indicators of urban BOW through their spectral characteristics and differentiation. This research combined ground in-situ water quality data with ground hyperspectral data collected from main urban BOWs in Guangzhou, China, and integrated factorial data mining and machine learning techniques to investigate how to monitor urban BOW. Eight key water quality parameters at 52 sample sites were used to retrieve three latent dimensions of urban BOW quality by factorial data mining. The synchronically measured hyperspectral bands along with the band combinations were examined by the machine learning technique, Lasso regression, to identify the most correlated bands and band combinations, over which three multiple regression models were fitted against three latent water quality indicators to determine which spectral bands were highly sensitive to three dimensions of urban BOW pollution. The findings revealed that the many sensitive bands were concentrated in higher hyperspectral band ranges, which supported the unique contribution of hyperspectral data for monitoring water quality. In addition, this integrated data mining and machine learning approach overcame the limitations of conventional band selection, which focus on a limited number of band ratios, band differences, and reflectance bands in the lower range of infrared region. The outcome also indicated that the integration of dimensionality reduction with feature selection shows good potential for monitoring urban BOW. This new analysis framework can be used in urban BOW monitoring and provides scientific data for policymakers to monitor it.

Introduction

The volume of industrial wastewater and urban domestic sewage discharged into human living spaces and the natural environment has increased with economic and industrial development and urban growth. Among various types of pollution, water pollution is one of the most considerable threats to socioeconomic sustainability due to its tremendous hazard to human health and environmental health (Song et al., 2019). In developing countries experiencing rapid urbanization, urban water pollution is a leading concern due to inadequate wastewater treatment facilities and regulations (Water Pollution Worries in Developing World, 2020). For instance, in developing countries such as Nepal, India, and Bangladesh, enormous amounts of pollutants discharged in urban areas cause critical river pollution near urban vicinities (Karn et al., 2001). The main cause of surface and groundwater pollution is the discharge of industrial and urban residential wastewater into rivers or underground aquifers (Song et al., 2017). When water is polluted, it is often dark/black in color and gives off smelly gases and is thus known as black odorous water (BOW) (Wang et al., 2014).

The attention to urban BOW can be traced back to the early-mid-1980s and the “The Great Stink” problem reported in the Thames River, United Kingdom (Mann, 2016). Since then, urban BOW phenomena have been observed and studied globally. The population in Mascow Russia more than doubled from the 1970s to the 1980s. Discharging of wastewater into rivers posed a huge challenge and caused malodorous and unsanitary urban areas (Martin, 2008). Sado-Inamura and Fukushi (2018) quantitatively analyzed the odor of urban rivers in Japan by developing human olfaction-based order scores. During the last two decades, as the population rapidly rose and urbanized areas swiftly expanded, the frequencies and severities of BOW phenomena intensified dramatically in urban rivers. China is not an exception. The first incident of the urban BOW was reported in the Huangpu River of China in the early 1980s (Gu et al., 1983). In recent years, BOW was reported in most of the major rivers in China on a quarterly and annual basis (Wang and Yang, 2016). There is an urgent need to address the concerns of BOW pollution (Yu et al., 2020).

Urban BOW phenomena are caused by different physical and chemical processes and reflect unique physical and chemical characteristics (Liu et al., 2003; Gaafar et al., 2020). However, there are no unified evaluation methods or standards for detecting urban BOW phenomena due to the complexity of pollution sources and their occurrence in disparate regions (Liang et al., 2017; Chen et al., 2018). In most water quality studies, researchers determine the health status of a river by using a water quality index (Hasan et al., 2015). Common water quality indicators include dissolved oxygen (DO), chemical oxygen demand (COD), ammonium (NH₃-N), total nitrogen (TN), total phosphorus (TP), and total suspended solids (TSS) (Hasan et al., 2015; Wu et al., 2018; Yan et al., 2015). The Chinese Water Quality Standard identifies that three nutrient related indicators TN, TP, and chlorophyll (Chl) are the primary cause of eutrophication (Su et al., 2017). Meanwhile, Chl is selected as a criterion indicator, as it is closely related to TP (Varol, 2020). The increasing concentration of Chl causes many kinds of algae to grow in water and increases the absorption rate of water, thereby resulting in changes in spectral reflectance (Zhao et al., 2013). Furthermore, COD and biochemical oxygen demand (BOD₅) are thought to relate closely to the organic pollutants (Mohtar et al., 2019). In addition, TSS is a good indicator of water pollution caused by nutrients and metals and also plays a key role in physical and aesthetic degradation (Packman et al., 1999). However, there are not many studies in the past to examine all of these indicators comprehensively. For instance, Ouyang et al. (2005) examined the river water quality and its main pollutants by analyzing physicochemical parameters DO, organic parameters COD, and major nutrient indices ammonium and TP. DO and biochemical oxygen demand (BOD) were investigated to analyze the unpleasant odor problem in urban rivers (Sado-Inamura et al., 2018). Our study includes all eight commonly analyzed water quality parameters, DO, NH₃-N, TP, TN, COD, BOD₅, TSS, and Chl.

Current urban BOW studies heavily rely on traditional field water sampling and testing approaches, but seldom apply newly developed remote sensing technologies to monitor BOW pollution (Shen et al., 2017). However, regarding water quality management, it is a feasible and economic approach to use multispectral remote sensing images and remote sensing techniques to study water pollution (Gan et al., 2010; Vakili and Amanollahi, 2020). In addition, researchers have studied water pollution issues and developed water quality models using various spectroradiometers. They differ by resolution from medium to high and are often used to gather diverse spectral responses for inland water sites with differing concentrations of water quality parameters (Hestir et al., 2015; Jay et al., 2017; Su, 2017; Xu et al., 2018). Moreover, statistical methods are often used to examine relationships between in situ concentration data of water quality parameters and remote sensing reflectance data. Ekercin (2007) examined the water quality of an inland river based on high-resolution multispectral images, and accurately interpreted the water quality condition of the study area. Because of the immensely narrow bandwidth, which is a significant advantage of hyperspectral remote sensing data, slight radiometric differences of the target can be observed and quantified. Jiao et al. (2006) investigated in-water constitution of Chl through hyperspectral remote sensing data. Xu et al. (2018) studied chromophoric dissolved organic matter (CODM) absorption using in-situ hyperspectral reflectance data and water samples. Shafique et al. (2003) used remotely sensed spectral bands and in-situ data to study chlorophyll and phosphorus and how they were influencing the turbidity of water.

It is particularly useful to detect water pollution severity using hyperspectral reflectance data with relatively narrow bandwidth to analyze BOW pollution (Thiemanna and Kaufmannb, 2002). The Chinese high-resolution remote sensing satellite Gaofen-2 (GF-2, 0.8 m) was analyzed to study urban BOW phenomena and to develop several retrieval algorithms with high accuracy (Shen et al., 2019). Another interesting study used Landsat 8 OLI and Sentinel-2 MSI imageries to identify the black lake issue by retrieving water quality indicators (Kuster et al., 2016). Their findings indicate that remote sensing is especially useful in identifying polluted black lakes. It should also be noted that water pollution remote sensing in recent years has integrated a new optimal classification method with data regression analysis to upgrade the remote sensing inversion model of TP (Du et al., 2018). In brief, remote sensing presents a complementary approach for studying comprehensive BOW in urban environments (Shen et al., 2017).

The narrow reflectance bands are an advantage of hyperspectral remote sensing when studying the urban BOW phenomena (Thiemanna and Kaufmannb, 2002). On the other hand, they also increase the difficulty of identifying corresponding relationships between reflectance bands and different water quality indicators due to a substantially large number of bands involved. Band selection is commonly done by examining high spectral correlations with the variables of interest (Du and Yang, 2008). The vast data volume could cause compounding or inaccurate outcomes if correlations between hundreds of bands were conducted. Traditionally four main types of algorithms are applied to examine spectral indication of water pollution: 1) single band-ratio empirical algorithm, 2) semi-empirical baseline algorithm, 3) multiple bands linear regression algorithm, and 4) semi-analytical band-ratio algorithm (Mishra et al., 2013). For instance, Randolph et al. (2008) studied the optimal active pigments chlorophyll and phycocyanin in turbid productive water based on band ratios 709nm/665 nm and 709nm/620 nm, which were initially proposed by Hair et al. (1998) and Allison (1999). Xu et al. (2018) selected the band ratios of 767nm826nm and767nm/826 nm to establish a model for clean water and turbid water, respectively. The band-ratio algorithm is widely used because it can eliminate the influence of environmental conditions to improve the accuracy of the analysis. However, the band-ratio algorithm has the disadvantage of spectrally related reflectance error, which could be overcome by the band-difference algorithm (Lee et al., 2002). However, both the band-ratio and band-difference methods simply focus on a very limited number of bands.

In recent studies, researchers have analyzed water quality parameters and their relations by applying machine learning techniques (Haghiabi et al., 2018). To overcome the uncertainty of the estimation model of a single water parameter, machine learning algorithms are implemented to assess water quality with higher accuracy (Wang et al., 2017). Machine learning is a sub-field of computer science that focuses on algorithms for learning from data and, consequently, identifies highly representative features to make predictions for new unseen data (Yang et al., 2018). The goal of using machine learning technology on the urban BOW issue is to overcome the conventional band selection methods, which highly rely on empirical band combinations.

Although there is a growing literature dealing with applications of hyperspectral remote sensing and machine learning techniques for assessing water quality, no comprehensive or integrated analytical frameworks for measuring urban BOW have been reported. The purpose of this study is to develop an integrated analytical framework combining hyperspectral remote sensed data, statistical analysis and machine learning to identify an effective approach for examining relationships between water quality parameters and urban BOW, for selecting spectral bands that are most sensitive to BOW, and for validating the outcome. Three rivers in Guangzhou City of China were chosen as the study sites to confirm this analytical framework. The study area, data and methods will be presented in Section 2. The results will be explained and discussed in Section 3. The conclusions will be given in Section 4.