Rainfall retrieval using commercial microwave links: Effect of sampling strategy on retrieval accuracy
Introduction
Accurate and timely rainfall information is crucial for real-time flood forecasting and various agricultural management activities. Wherever available, dense rain gauge networks combined with operational weather radars are currently considered as the most reliable source of temporal and spatial rainfall estimates at ground level (Russell et al., 2010). However, deployment of such infrastructure for national coverage is costly, and most low-to middle-income countries are not able to afford such equipment. Moreover, there are a number of limitations with this rainfall measurement approach. For example, conventional rain gauges provide only discrete point observations which may be biased due to wind and splash effects(Muller and Kidd, 2006). While radar rainfall products provide integrated observations with a large spatial coverage, they do not directly measure rainfall close to the ground. Instead, their measurements are based on an indirect measurement of reflected microwave energy from hundreds of meters above the ground making them subject to large uncertainties in their retrieval. These include hardware calibration, beam broadening, attenuation due to rain, ground clutter, anomalous propagation, and wind effects (Doviak, 1984, Joss et al., 1990, Germann et al., 2006, Berne and Krajewski, 2013). Alternatively, satellites provide a nearly global spatial coverage of rainfall, but they do not provide a direct measurement of rainfall near the ground either. Rather, they provide an indirect estimate based on cloud top temperatures or emitted or reflected microwave energy. Thus these data still need to be calibrated using ground measurements, which limits the stand-alone use of satellite products for operational applications (Kucera et al., 2013).
In contrast to the traditional methods outlined above, commercial microwave links (CMLs) operated by mobile network operators (MNOs) have proven to be a complementary source of rainfall information (Messer, 2018, Uijlenhoet et al., 2018, Chwala and Kunstmann, 2019). A CML includes a transmitter on one backhaul tower and a receiver on another backhaul tower. Depending on the number of transmitters and receivers on these towers, a CML could include a single link or multiple links at the same or different frequencies and/or polarisations, known as “sub-links”. The commonly called “duplex links” are a CML that includes two sub-links permitting two-way data transmission. In addition, redundant links may be installed to back up the existing operational link during a failure. The primary objective of these CMLs is to provide telecommunication services, but the information they collect to maintain their network could be utilized for retrieving rainfall rates (Messer et al., 2006, Leijnse et al., 2007).This technique of rainfall measurement has been proven useful in areas where there are no other sources of rainfall information, but also in densely populated cities where the existing infrastructure does not provide reliable means of observation (Overeem et al., 2011, Pastorek et al., 2019). In such cities, it is usually problematic to find the appropriate location to install rain gauges among high-rise buildings according to the official World Meteorological Organisation requirements, with the high-rise buildings also creating ground clutter for weather radars. The CMLs which are widespread in urban areas typically transmit the microwave signals over distances of a couple of hundred meters to a few tens of kilometres, at tens of meters above the ground, thus providing path-integrated rainfall information close to the ground (Overeem et al., 2016b, Gazit and Messer, 2018).
CML-derived rainfall estimates are based on the fact that the transmitted signal is attenuated as it passes through the rain medium. This attenuation in the signal is more pronounced at higher frequency bands, e.g. it typically becomes significant above 10 GHz. This phenomenon was widely studied by the telecommunication engineering community in the early 1960 s to design an optimal spacing between microwave towers for efficient and reliable communication (Hogg, 1968). Later, various experimental studies using microwave links showed that inversion of the technique could be used for rainfall retrieval (Atlas and Ulbrich, 1977, Giuli et al., 1991, Christopher et al., 1996, Mello et al., 2002, Holt et al., 2003). However, application of this technique was limited until the early 2000 s, when Messer et al., 2006, Leijnse et al., 2007 concomitantly demonstrated the use of CML signal attenuation for rainfall measurement. This was a major breakthrough toward demonstrating the potential to use the more than 4 million commercial microwave links in the world (Ericsson, 2017) for rainfall monitoring purposes. Subsequently, this technique gained popularity with feasibility and validation studies undertaken for a variety of locations around the world including: Brazil (Rios Gaona et al., 2015), Burkina Faso (Doumounia et al., 2014), Czech Republic (Fencl et al., 2013, Fencl et al., 2017), Germany (Chwala et al., 2012, Chwala et al., 2016, Smiatek et al., 2017, Graf et al., 2020), Israel (Messer et al., 2006, Goldshtein et al., 2009), Italy (Roversi et al., 2020) , The Netherlands (Leijnse et al., 2007, Overeem et al., 2011, Overeem et al., 2013, Overeem et al., 2016b, de Vos et al., 2019), Pakistan (Sohail Afzal et al., 2018) and Switzerland (Bianchi et al., 2013).
These validation studies have been conducted based on a few links to a couple of thousand links covering an entire country such as The Netherlands (Overeem et al., 2013) and Germany (Graf et al., 2020). The temporal resolution of such CML rainfall estimates typically varies from a few seconds to 15 min, with most telecommunication operators sampling the received signal level (RSL) at 10 Hz but storing it at a much coarser temporal resolution. In most studies, 15-minute minimum and maximum RSL data, as stored operationally by the MNO’s network management systems, were used for rainfall retrieval (Leijnse et al., 2007, Goldshtein et al., 2009, Overeem et al., 2011, Overeem et al., 2016b, Rios et al., 2017). There have been a few studies using 1-min and even higher temporal resolution, up to a second, instantaneous RSL data for rainfall estimation (Doumounia et al., 2014, Chwala et al., 2016). Overeem et al. (2016b) evaluated 2.5 years of microwave link rainfall estimates for the Netherlands with more than 3000 microwave links (using 15-minute minimum–maximum sampling) against gauge-adjusted radar rainfall data, showing a relative underestimation of 9% for 15-min interpolated rainfall maps with a 74 km2 resolution. However, the interpolated hourly rainfall map using CMLs outperformed automatic rain gauges compared with gauge adjusted radar data. Similarly, Chwala et al. (2012) used 1-minute averaged RSL data recorded with data loggers for five microwave links, showing a good correlation between link and radar-derived rainfall.
Some of the MNOs also provide instantaneous RSL (periodic snapshots) data over the 15-minutes: de Vos et al. (2019) compared the performance of instantaneous versus minimum and maximum RSL data for The Netherlands. Even though this comparison was based on data from two different periods, each having a different network, the use of minimum and maximum sampled data outperformed the instantaneous 15-minute data. Similarly, average sampling of the received signal level over the 15-minute interval is also common for telecommunication operators in some parts of the world, but this has not been evaluated against the widely used minimum and maximum RSL strategy. Accordingly, this study tests this alternative strategy, while demonstrating for the first time the capability of rainfall retrieval using CML signal attenuation data in the Australian continent
To date, there has not been a study evaluating the errors introduced by the minimum–maximum sampling as opposed to average sampling. This study explores the capability of rainfall retrieval using CML signal attenuation data for the Greater Metropolitan area of Melbourne, the second largest city in Australia, with a population of 4.48 million (Australian Bureau of Statistics, 2016). This study compares the performance of rainfall retrieval using two commonly sampled datasets for the same period, where data based on minimum–maximum and average sampling from the same link paths are compared. A total of 135 microwave links are used, covering approximately 2 years of data. These CML data were stored by the network monitoring system (NMS) every 15-minute based on10 Hz sampling data. This 15-minute data includes the minimum, maximum and average over 15-min intervals with the constant transmitted power.
Section snippets
Description of the study area
The study area covers the greater Melbourne region in the Australian state of Victoria. This region has a temperate oceanic climate (Cfb, Köppen-Geiger classification), with an annual average rainfall (based on 29 years of rainfall data from 1990 until 2018 for 73 stations) varying from 500 mm in the west of Melbourne to 1400 mm in the Dandenong ranges towards the eastern part of the city, with a standard deviation of 175 mm. Most of the rainfall occurs during southern hemisphere winter (June,
Preliminary data processing and quality check
The CML dataset was delivered by the operator in two separate files: one with 15-min RSL data for all the links stored daily, and the other with the corresponding metadata. These files were received on a monthly basis at the end of each month. Metadata included the location of transmitter and receiver nodes, the elevation of the antennas, the assigned microwave frequencies (including frequency bandwidth), the polarization of the signal, path lengths and the IP addresses of each transmitter and
Calibration
Table 3 shows the calibration results for the four most important RAINLINK parameters. Two parameters, threshold median and threshold median per unit length are related to the wet-dry classification while the remaining two, wet antenna attenuation ( and Alpha (α), are related to rainfall retrievals. For Average, both and are less negative compared to the default values of RAINLINK while the value is also lower compared to MinMax, but higher compared to
Validation result
After processing the dataset (excluding the data used for calibration), the number of data points containing results for the two sampling strategies (MinMax and Average) were of different sizes, because the outlier filter used in processing the raw data removed different time intervals for specific links from MinMax and Average data. Thus, in order to make a fair comparison, the time intervals with available data for both strategies were retained. Also, to note that all the performance
Optimized parameters for RAINLINK
Among the parameters used in RAINLINK, only the four most sensitive parameters were optimized for this Melbourne dataset, specifically the threshold median, the threshold median per unit length , wet antenna attenuation ( for Average RSL data and the alpha (α) for MinMax RSL data. In addition, two parameters, a and b, were obtained based on local disdrometer data from the study of Guyot et al. (2020), and are the most critical for rainfall retrieval of all the parameters. In the
Conclusion
This study presents rainfall retrievals over the greater Melbourne Metropolis using 135 commercial microwave links operating at frequencies ranging from 10 to 40 GHz and path lengths of 0.2 km to 25 km. This study is the very first to conduct a comparison of rainfall retrieval using CML data with two different sampling strategies over the same link paths (average and min/max RSL data over 15 min). For this, the RAINLINK package was used and a new set of parameters was derived for rainfall
Software and model codes
Rainfall retrieval was undertaken using the R package RAINLINK version 1.2, which is freely available on GitHub (https://github.com/overeem11/RAINLINK). All other data processing and plotting was done using Python.
Data availability
The CML data is not publicly available due to a non-disclosure agreement. Other data are available upon request.
CRediT authorship contribution statement
Jayaram Pudashine: Conceptualization, Methodology, Software, Formal analysis, Investigation, Writing – original draft. Adrien Guyot: Methodology, Software, Writing - review & editing, Supervision. Aart Overeem: Methodology, Software, Writing - review & editing. Valentijn R.N. Pauwels: Writing - review & editing, Supervision. Alan Seed: Writing - review & editing. Remko Uijlenhoet: Methodology, Writing - review & editing. Mahesh Prakash: Writing - review & editing. Jeffrey P. Walker: Writing -
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
An Australian Research Council Discovery Project (Grant number DP160101377) supported this project. The authors also acknowledge the Commonwealth Scientific and Industrial Research Organisation (CSIRO) for the top-up scholarship for the first author of his study. The authors would like to thank Motorola, the Bureau of Meteorology (BoM), and Melbourne Water for providing data. The authors declare no conflict of interest.
References (50)
- et al.
Radar for hydrology: Unfulfilled promise or unrecognized potential?
Adv. Water Resour.
(2013) - et al.
Quality control of rain gauge measurements using telecommunication microwave links
J. Hydrol.
(2013) Comparison of the predicted and observed secondary structure of T4 phase lysozyme
BBA
(1975)- et al.
T-matrix computations of light scattering by large spheroidal particles
Opt. Commun.
(1994) - et al.
Commercial microwave links for urban drainage modelling: The effect of link characteristics and their position on runoff simulations
J. Environ. Manage.
(2019) - Australian Bureau of Statistics:...
- et al.
Path-and Area-Integrated Rainfall Measurement by Microwave Attenuation in the 1–3 cm Band
J. Appl. Meteorol.
(1977) - et al.
CORRIGENDUM: False Alarm Rate or False Alarm Ratio?
Weather Forecas.
(2009) - et al.
35-GHz Dual-Polarization Propagation Link for Rain-Rate Estimation
J. Atmos. Oceanic Technol.
(1996) - et al.
Precipitation observation using microwave backhaul links in the alpine and pre-alpine region of Southern Germany
Hydrol. Earth Syst. Sci.
(2012)
Real-time data acquisition of commercial microwave link networks for hydrometeorological applications
Atmos. Meas. Tech.
Commercial microwave link networks for rainfall observation: Assessment of the current status and future challenges, Wiley Interdisciplinary Reviews
Water
Rainfall Estimation Accuracy of a Nationwide Instantaneously Sampling Commercial Microwave Link Network: Error Dependency on Known Characteristics
J. Atmos. Oceanic Technol.
Rainfall monitoring based on microwave links from cellular telecommunication networks: First results from a West African test bed
Geophys. Res. Lett.
Gauge-adjusted rainfall estimates from commercial microwave links
Hydrol. Earth Syst. Sci.
Assessing the potential of using telecommunication microwave links in urban drainage modelling, Water science and technology : a journal of the International Association on Water
Pollution Research
Advancements in the Statistical Study, Modeling, and Simulation of Microwave-Links in Cellular Backhaul Networks
Environments
Radar precipitation measurement in a mountainous region
Q. J. R. Meteorolog. Soc.
Tomographic reconstruction of rainfall fields through microwave attenuation measurments
J. Appl. Meterol.
Rain Rate Estimation Using Measurements From Commercial Telecommunications Links
IEEE Trans. Signal Process.
Rainfall estimation from a German-wide commerical microwave link network : Optimized processing and validation for 1 year of data
Hydrol. Earth Syst. Sci.
Effect of disdrometer type on rain drop size distribution characterisation: a new dataset for south-eastern Australia
Hydrol. Earth Syst. Sci.
Millimeter-Wave Communication through the Atmosphere: The known and unknown features in propagation of short radio waves are discussed
Science (New York, N.Y.)
Cited by (10)
Traditional and Novel Methods of Rainfall Observation and Measurement: A Review
2023, Journal of Hydrometeorology