Rainfall retrieval using commercial microwave links: Effect of sampling strategy on retrieval accuracy

https://doi.org/10.1016/j.jhydrol.2021.126909Get rights and content

Highlights

  • The first validation study of using commercial microwave links for rainfall retrieval in the Australian continent.

  • The parameters of an open-source algorithm named RAINLINK were calibrated using the new Melbourne dataset.

  • Rainfall retrieval based on two commonly adopted sampling strategies were compared.

  • The overall analysis showed that 15-minute minimum and maximum sampling strategy outperformed 15-minute average sampling strategy.

Abstract

This study presents the first evaluation of using commercial microwave link (CML) data for rainfall measurements in Australia, with the test site being the greater Melbourne Metropolitan area. More than 100 CMLs with microwave frequency ranging between 10 and 40 GHz have been used for the rainfall retrieval. The 15-minute received signal levels (RSLs) for each CML based on two sampling strategies (average and minimum/maximum) collected for 2 years provided a unique dataset to compare performances of rainfall retrievals. The open source algorithm RAINLINK was used for deriving rainfall from the 15-minute RSL data. From two years of data, a subset of 30 rainy days distributed across this period were used for calibrating the RAINLINK parameters, with the remaining data used for validation. For this study, only path-averaged rainfall intensities were validated based on a gauge-adjusted radar product serving as the reference. The result of the wet-dry classification showed that the minimum and maximum RSL data performed better, with lower probability of false detection and higher Matthews correlation coefficient than average RSL data. For the rainfall retrieval, both datasets showed similar correlation with the gauge adjusted radar product. However, based on other statistics (RMSE, bias and CV) minimum and maximum RSL data outperformed average for the rainfall retrieval. Overall, this study highlights the robust accuracy of commercial microwave links for rainfall retrieval while using only minimum and maximum RSL data.

Introduction

Accurate and timely rainfall information is crucial for real-time flood forecasting and various agricultural management activities. Wherever available, dense rain gauge networks combined with operational weather radars are currently considered as the most reliable source of temporal and spatial rainfall estimates at ground level (Russell et al., 2010). However, deployment of such infrastructure for national coverage is costly, and most low-to middle-income countries are not able to afford such equipment. Moreover, there are a number of limitations with this rainfall measurement approach. For example, conventional rain gauges provide only discrete point observations which may be biased due to wind and splash effects(Muller and Kidd, 2006). While radar rainfall products provide integrated observations with a large spatial coverage, they do not directly measure rainfall close to the ground. Instead, their measurements are based on an indirect measurement of reflected microwave energy from hundreds of meters above the ground making them subject to large uncertainties in their retrieval. These include hardware calibration, beam broadening, attenuation due to rain, ground clutter, anomalous propagation, and wind effects (Doviak, 1984, Joss et al., 1990, Germann et al., 2006, Berne and Krajewski, 2013). Alternatively, satellites provide a nearly global spatial coverage of rainfall, but they do not provide a direct measurement of rainfall near the ground either. Rather, they provide an indirect estimate based on cloud top temperatures or emitted or reflected microwave energy. Thus these data still need to be calibrated using ground measurements, which limits the stand-alone use of satellite products for operational applications (Kucera et al., 2013).

In contrast to the traditional methods outlined above, commercial microwave links (CMLs) operated by mobile network operators (MNOs) have proven to be a complementary source of rainfall information (Messer, 2018, Uijlenhoet et al., 2018, Chwala and Kunstmann, 2019). A CML includes a transmitter on one backhaul tower and a receiver on another backhaul tower. Depending on the number of transmitters and receivers on these towers, a CML could include a single link or multiple links at the same or different frequencies and/or polarisations, known as “sub-links”. The commonly called “duplex links” are a CML that includes two sub-links permitting two-way data transmission. In addition, redundant links may be installed to back up the existing operational link during a failure. The primary objective of these CMLs is to provide telecommunication services, but the information they collect to maintain their network could be utilized for retrieving rainfall rates (Messer et al., 2006, Leijnse et al., 2007).This technique of rainfall measurement has been proven useful in areas where there are no other sources of rainfall information, but also in densely populated cities where the existing infrastructure does not provide reliable means of observation (Overeem et al., 2011, Pastorek et al., 2019). In such cities, it is usually problematic to find the appropriate location to install rain gauges among high-rise buildings according to the official World Meteorological Organisation requirements, with the high-rise buildings also creating ground clutter for weather radars. The CMLs which are widespread in urban areas typically transmit the microwave signals over distances of a couple of hundred meters to a few tens of kilometres, at tens of meters above the ground, thus providing path-integrated rainfall information close to the ground (Overeem et al., 2016b, Gazit and Messer, 2018).

CML-derived rainfall estimates are based on the fact that the transmitted signal is attenuated as it passes through the rain medium. This attenuation in the signal is more pronounced at higher frequency bands, e.g. it typically becomes significant above 10 GHz. This phenomenon was widely studied by the telecommunication engineering community in the early 1960 s to design an optimal spacing between microwave towers for efficient and reliable communication (Hogg, 1968). Later, various experimental studies using microwave links showed that inversion of the technique could be used for rainfall retrieval (Atlas and Ulbrich, 1977, Giuli et al., 1991, Christopher et al., 1996, Mello et al., 2002, Holt et al., 2003). However, application of this technique was limited until the early 2000 s, when Messer et al., 2006, Leijnse et al., 2007 concomitantly demonstrated the use of CML signal attenuation for rainfall measurement. This was a major breakthrough toward demonstrating the potential to use the more than 4 million commercial microwave links in the world (Ericsson, 2017) for rainfall monitoring purposes. Subsequently, this technique gained popularity with feasibility and validation studies undertaken for a variety of locations around the world including: Brazil (Rios Gaona et al., 2015), Burkina Faso (Doumounia et al., 2014), Czech Republic (Fencl et al., 2013, Fencl et al., 2017), Germany (Chwala et al., 2012, Chwala et al., 2016, Smiatek et al., 2017, Graf et al., 2020), Israel (Messer et al., 2006, Goldshtein et al., 2009), Italy (Roversi et al., 2020) , The Netherlands (Leijnse et al., 2007, Overeem et al., 2011, Overeem et al., 2013, Overeem et al., 2016b, de Vos et al., 2019), Pakistan (Sohail Afzal et al., 2018) and Switzerland (Bianchi et al., 2013).

These validation studies have been conducted based on a few links to a couple of thousand links covering an entire country such as The Netherlands (Overeem et al., 2013) and Germany (Graf et al., 2020). The temporal resolution of such CML rainfall estimates typically varies from a few seconds to 15 min, with most telecommunication operators sampling the received signal level (RSL) at 10 Hz but storing it at a much coarser temporal resolution. In most studies, 15-minute minimum and maximum RSL data, as stored operationally by the MNO’s network management systems, were used for rainfall retrieval (Leijnse et al., 2007, Goldshtein et al., 2009, Overeem et al., 2011, Overeem et al., 2016b, Rios et al., 2017). There have been a few studies using 1-min and even higher temporal resolution, up to a second, instantaneous RSL data for rainfall estimation (Doumounia et al., 2014, Chwala et al., 2016). Overeem et al. (2016b) evaluated 2.5 years of microwave link rainfall estimates for the Netherlands with more than 3000 microwave links (using 15-minute minimum–maximum sampling) against gauge-adjusted radar rainfall data, showing a relative underestimation of 9% for 15-min interpolated rainfall maps with a 74 km2 resolution. However, the interpolated hourly rainfall map using CMLs outperformed automatic rain gauges compared with gauge adjusted radar data. Similarly, Chwala et al. (2012) used 1-minute averaged RSL data recorded with data loggers for five microwave links, showing a good correlation between link and radar-derived rainfall.

Some of the MNOs also provide instantaneous RSL (periodic snapshots) data over the 15-minutes: de Vos et al. (2019) compared the performance of instantaneous versus minimum and maximum RSL data for The Netherlands. Even though this comparison was based on data from two different periods, each having a different network, the use of minimum and maximum sampled data outperformed the instantaneous 15-minute data. Similarly, average sampling of the received signal level over the 15-minute interval is also common for telecommunication operators in some parts of the world, but this has not been evaluated against the widely used minimum and maximum RSL strategy. Accordingly, this study tests this alternative strategy, while demonstrating for the first time the capability of rainfall retrieval using CML signal attenuation data in the Australian continent

To date, there has not been a study evaluating the errors introduced by the minimum–maximum sampling as opposed to average sampling. This study explores the capability of rainfall retrieval using CML signal attenuation data for the Greater Metropolitan area of Melbourne, the second largest city in Australia, with a population of 4.48 million (Australian Bureau of Statistics, 2016). This study compares the performance of rainfall retrieval using two commonly sampled datasets for the same period, where data based on minimum–maximum and average sampling from the same link paths are compared. A total of 135 microwave links are used, covering approximately 2 years of data. These CML data were stored by the network monitoring system (NMS) every 15-minute based on10 Hz sampling data. This 15-minute data includes the minimum, maximum and average over 15-min intervals with the constant transmitted power.

Section snippets

Description of the study area

The study area covers the greater Melbourne region in the Australian state of Victoria. This region has a temperate oceanic climate (Cfb, Köppen-Geiger classification), with an annual average rainfall (based on 29 years of rainfall data from 1990 until 2018 for 73 stations) varying from 500 mm in the west of Melbourne to 1400 mm in the Dandenong ranges towards the eastern part of the city, with a standard deviation of 175 mm. Most of the rainfall occurs during southern hemisphere winter (June,

Preliminary data processing and quality check

The CML dataset was delivered by the operator in two separate files: one with 15-min RSL data for all the links stored daily, and the other with the corresponding metadata. These files were received on a monthly basis at the end of each month. Metadata included the location of transmitter and receiver nodes, the elevation of the antennas, the assigned microwave frequencies (including frequency bandwidth), the polarization of the signal, path lengths and the IP addresses of each transmitter and

Calibration

Table 3 shows the calibration results for the four most important RAINLINK parameters. Two parameters, threshold median (QmP) and threshold median per unit length (QmPL), are related to the wet-dry classification while the remaining two, wet antenna attenuation (Aa) and Alpha (α), are related to rainfall retrievals. For Average, both QmP and QmPL are less negative compared to the default values of RAINLINK while the Aa value is also lower compared to MinMax, but higher compared to

Validation result

After processing the dataset (excluding the data used for calibration), the number of data points containing results for the two sampling strategies (MinMax and Average) were of different sizes, because the outlier filter used in processing the raw data removed different time intervals for specific links from MinMax and Average data. Thus, in order to make a fair comparison, the time intervals with available data for both strategies were retained. Also, to note that all the performance

Optimized parameters for RAINLINK

Among the parameters used in RAINLINK, only the four most sensitive parameters were optimized for this Melbourne dataset, specifically the threshold median(QmP), the threshold median per unit length (QmPL), wet antenna attenuation (Aa) for Average RSL data and the alpha (α) for MinMax RSL data. In addition, two parameters, a and b, were obtained based on local disdrometer data from the study of Guyot et al. (2020), and are the most critical for rainfall retrieval of all the parameters. In the

Conclusion

This study presents rainfall retrievals over the greater Melbourne Metropolis using 135 commercial microwave links operating at frequencies ranging from 10 to 40 GHz and path lengths of 0.2 km to 25 km. This study is the very first to conduct a comparison of rainfall retrieval using CML data with two different sampling strategies over the same link paths (average and min/max RSL data over 15 min). For this, the RAINLINK package was used and a new set of parameters was derived for rainfall

Software and model codes

Rainfall retrieval was undertaken using the R package RAINLINK version 1.2, which is freely available on GitHub (https://github.com/overeem11/RAINLINK). All other data processing and plotting was done using Python.

Data availability

The CML data is not publicly available due to a non-disclosure agreement. Other data are available upon request.

CRediT authorship contribution statement

Jayaram Pudashine: Conceptualization, Methodology, Software, Formal analysis, Investigation, Writing – original draft. Adrien Guyot: Methodology, Software, Writing - review & editing, Supervision. Aart Overeem: Methodology, Software, Writing - review & editing. Valentijn R.N. Pauwels: Writing - review & editing, Supervision. Alan Seed: Writing - review & editing. Remko Uijlenhoet: Methodology, Writing - review & editing. Mahesh Prakash: Writing - review & editing. Jeffrey P. Walker: Writing -

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

An Australian Research Council Discovery Project (Grant number DP160101377) supported this project. The authors also acknowledge the Commonwealth Scientific and Industrial Research Organisation (CSIRO) for the top-up scholarship for the first author of his study. The authors would like to thank Motorola, the Bureau of Meteorology (BoM), and Melbourne Water for providing data. The authors declare no conflict of interest.

References (50)

  • C. Chwala et al.

    Real-time data acquisition of commercial microwave link networks for hydrometeorological applications

    Atmos. Meas. Tech.

    (2016)
  • C. Chwala et al.

    Commercial microwave link networks for rainfall observation: Assessment of the current status and future challenges, Wiley Interdisciplinary Reviews

    Water

    (2019)
  • L.W. de Vos et al.

    Rainfall Estimation Accuracy of a Nationwide Instantaneously Sampling Commercial Microwave Link Network: Error Dependency on Known Characteristics

    J. Atmos. Oceanic Technol.

    (2019)
  • A. Doumounia et al.

    Rainfall monitoring based on microwave links from cellular telecommunication networks: First results from a West African test bed

    Geophys. Res. Lett.

    (2014)
  • Doviak, R. J.: Doppler radar and weather observations, In: Zrnić, D. a. S., Orlando, Fla. (Eds.) Academic Press,...
  • Ericsson: Ericsson microwave outlook: trends and needs in the microwave industry,...
  • M. Fencl et al.

    Gauge-adjusted rainfall estimates from commercial microwave links

    Hydrol. Earth Syst. Sci.

    (2017)
  • M. Fencl et al.

    Assessing the potential of using telecommunication microwave links in urban drainage modelling, Water science and technology : a journal of the International Association on Water

    Pollution Research

    (2013)
  • L. Gazit et al.

    Advancements in the Statistical Study, Modeling, and Simulation of Microwave-Links in Cellular Backhaul Networks

    Environments

    (2018)
  • U. Germann et al.

    Radar precipitation measurement in a mountainous region

    Q. J. R. Meteorolog. Soc.

    (2006)
  • D. Giuli et al.

    Tomographic reconstruction of rainfall fields through microwave attenuation measurments

    J. Appl. Meterol.

    (1991)
  • O. Goldshtein et al.

    Rain Rate Estimation Using Measurements From Commercial Telecommunications Links

    IEEE Trans. Signal Process.

    (2009)
  • M. Graf et al.

    Rainfall estimation from a German-wide commerical microwave link network : Optimized processing and validation for 1 year of data

    Hydrol. Earth Syst. Sci.

    (2020)
  • A. Guyot et al.

    Effect of disdrometer type on rain drop size distribution characterisation: a new dataset for south-eastern Australia

    Hydrol. Earth Syst. Sci.

    (2020)
  • D.C. Hogg

    Millimeter-Wave Communication through the Atmosphere: The known and unknown features in propagation of short radio waves are discussed

    Science (New York, N.Y.)

    (1968)
  • Cited by (10)

    View all citing articles on Scopus
    View full text