Improving the predictions of soil properties from VNIR–SWIR spectra in an unlabeled region using semi-supervised and active learning

Highlights

• The semi-supervised learning framework of LapSVR is presented.

• LapSVR outperforms other supervised techniques in VNIR-SWIR spectroscopy.

• The active learning framework is presented along with a novel strategy.

• Active learning identifies samples from the unknown region that ought to be labeled.

• The proposed methodologies statistically outperform their counterparts.

Abstract

Monitoring the status of the soil ecosystem to identify the spatio-temporal extent of the pressures exerted and mitigate the effects of climate change and land degradation necessitates the need for reliable and cost-effective solutions. To address this need, soil spectroscopy in the visible, near- and shortwave-infrared (VNIR–SWIR) has emerged as a viable alternative to traditional analytical approaches. To this end, large-scale soil spectral libraries coupled with advanced machine learning tools have been developed to infer the soil properties from the hyperspectral signatures. However, models developed from one region may exhibit diminished performance when applied to a new, unseen by the model, region due to the large and inherent soil variability (e.g. pedogenetical differences, diverse soil types etc.). Given an existing spectral library with labeled data and a new unlabeled region (i.e. where no soil samples are analytically measured) the question then becomes how to best develop a model which can more accurately predict the soil properties of the unlabeled region.

In this paper, a machine learning technique leveraging on the capabilities of semi-supervised learning which exploits the predictors’ distribution of the unlabeled dataset and of active learning which expertly selects a small set of data from the unlabeled dataset as a spiking subset in order to develop a more robust model is proposed. The semi-supervised learning approach is the Laplacian Support Vector Regression following the manifold regularization framework. As far as the active learning component is concerned, the pool-based approach is utilized as it best matches with the aforementioned use-case scenario, which iteratively selects a subset of data from the unlabeled region to spike the calibration set. As a query strategy, a novel machine learning–based strategy is proposed herein to best identify the spiking subset at each iteration. The experimental analysis was conducted using data from the Land Use and Coverage Area Frame Survey of 2009 which covered most of the then member-states of the European Union, and in particular by focusing on the mineral cropland soil samples from 5 different countries. The statistical analysis conducted ascertained the efficacy of our approach when compared to the current state-of-the-art in soil spectroscopy.

Introduction

Soil is an important non-renewable resource that provides for agriculture, improves water quality, and buffers greenhouse gases in the atmosphere. The preservation and sustainable management of soils is crucial to tackle the main challenges that humanity is facing such as increasing demands for food by an increasing population, climate change, environmental degradation, water scarcity, and loss of biodiversity (Hatfield and Walthall, 2015, Montanarella et al., 2016, Hatfield et al., 2017, Amundson et al., 2015). Soil-related consequences of the aforementioned pressures include significant changes in soils properties, surface water and groundwater quality, food security, water supplies, human health, energy, agriculture, and the sustainability of ecosystems including that of the soil biota (Qafoku, 2015). The preservation of the soil ecosystem dictates the need for accurate information about the soil’s status in order to monitor its spatio-temporal variability. Assessing the state of the soil however requires complex analytical approaches which are costly in time and resource.

Soil spectroscopy in the visible, near-infrared and shortwave-infrared (VNIR–SWIR, 0.35 μm to 2.5 μm) has been established as a rapid and cost-efficient alternative to the traditional analytical methods required for the estimation of some key physical and chemical soil constituents (Stenberg et al., 2010, Nocita et al., 2015). In the past decades a number of large soil spectral libraries (SSLs) have been developed throughout the world and have meticulously recorded the VNIR–SWIR spectra, the physicochemical properties, and other metadata for thousands of soil samples (e.g. Rossel et al., 2016, Orgiazzi et al., 2018, Tziolas et al., 2019). These have enabled researchers to calibrate chemometric models that link the spectral signature to the known reference values of the soil properties, thus facilitating the operational application of soil spectroscopy. However, the application of models calibrated using data from one SSL to a new, unseen by the model, region or a local site remains a challenging task. This may occur because the unique characteristics of the new region are not appropriately encapsulated by the calibration set, thus leading to erroneous predictions by the model (Guerrero et al., 2010, Guerrero et al., 2014, Seidel et al., 2019).

Machine learning provides systems the ability to automatically learn and improve from experience. Supervised machine learning approaches rely on the identification of patterns from a set of labeled data, which are records consisting of a set of predictive features coupled with a target output (label) provided by an expert. In soil spectroscopy the predictive features are the diffuse reflectance spectral signatures while the labels are the soil properties. With the advent of artificial intelligence and deep learning techniques which can cope well with big data, many efforts have focused on using data comprised of hundreds of thousands of records. Supervised approaches like multi-kernel support vector machines (Tsakiridis et al., 2020), fuzzy rule-based systems (Tsakiridis et al., 2019), convolutional neural networks (Padarian et al., 2018, Tsakiridis et al., 2020), Gaussian processes (Ramirez-Lopez et al., 2013, Tziolas et al., 2019), and local partial least squares regression (Nocita et al., 2014) have exhibited their potential to deal with soil hyperspectral data. However, data labeling can be expensive both in terms of financial cost and labeling time in many applications (e.g. for speech recognition or medical imaging). This includes the problem tackled by soil spectroscopy where data labeling refers to the measurement of soil properties through analytical techniques in a chemical laboratory. In these applications it is common to have a few labeled and a plethora of unlabeled data. Semi-supervised learning aims at exploiting the prior knowledge from unlabeled data to enhance the performance of the learning algorithm, and it is particularly adept when the labeled data are few while the unlabeled data numerous. Some approaches of semi-supervised learning are based on the manifold assumption (Belkin et al., 2006) which states that the marginal distribution underlying the data can be described using a low-dimensional manifold embedded in a higher dimensional space. The Laplacian support vector machine (LapSVM) is a graph-based model following this assumption (Belkin et al., 2006).

Active learning on the other hand is a technique aiming at selecting the most informative samples from the pool of unlabeled data, which are subsequently labeled by a human or a machine annotator and thence incorporated into the calibration set. It is motivated by the observation that a model may enhance its performance while simultaneously use less labeled data for training, if it can choose the data on which it trains (Cohn et al., 1996). In scenarios where the labeled data are few, because the learner is able to choose the examples to be labeled, it can develop a model with substantially fewer data for training than required in normal supervised learning (with a priori labeling of the data). There have been various query strategies proposed in pool-based active learning, such as: (i) uncertainty sampling, where the samples for which the model is most uncertain are selected for labelling, (ii) query-by-committee, where a group of models predicts the unknown samples, and the samples on which they most disagree are selected for labelling, (iii) estimated error reduction, that estimates the expected future error which would have resulted if some instances were labeled and incorporated into the calibration set.

To account for the less accurate predictions at local or unseen sites various approaches have been proposed. They are mostly focused on (i) either constraining the SSL by selecting a subset of samples based on spectral or other similarities and then building a new model using only this subset of data (e.g. Nocita et al., 2014, Xu et al., 2016, Lobsey et al., 2017, Castaldi et al., 2018), or (ii) augmenting the SSL with site-specific samples (the so-termed spiking approach) (e.g. Sankey et al., 2008, Guerrero et al., 2010, Wetterlind and Stenberg, 2010, Jiang et al., 2017), or a combination thereof. In the first case, no data are labeled using analytical methods from the new site, which may not work well if the new site is not well-represented in the existing SSL. In the latter case, the most important question that arises is which and how many site-specific samples should be selected for labeling (Lucà et al., 2017, Nawar and Mouazen, 2018).

Our approach is motivated by this need to partially alleviate the cost of laboratory analyses which is needed for (i) accurate site-specific calibrations, and (ii) whilst establishing large SSLs. A particularly noteworthy application of this second case may be for the development of the soil system for Africa under the Soils4Africa project, which is funded by the European Union under the Horizon 2020 SFS-35–2019-2020 call, and will take place in the next four years (2020–2023). In particular, it may help identify which portion from the large number of soil samples that will be collected (ca. 20,000) can be very accurately predicted and which needs to be included in the calibration set based on its spectral properties, thus partially alleviating the costs associated with the analytical techniques.

The major contributions of this work are as follows:

• It explores the use of semi-supervised learning in the domain of soil spectroscopy to identify whether it can enhance the prediction accuracy compared to traditionally used supervised approaches.

• A novel machine learning–based strategy is proposed to select the samples from the unlabeled region that ought to be labeled (i.e. queried) in order to attain a more robust model and compared with existing strategies.

• It explores the use of active learning and compares its application in soil spectroscopy with other approaches proposed in the literature.

The rest of the paper is organized as follows. Section 2 briefly presents the semi-supervised Laplacian Support Vector Regression algorithm. Section 3 lays out the concepts of active learning and introduces the novel machine learning–based strategy proposed herein. In Section 4 the experimental set up is presented, including the dataset formulation, the algorithms used to compare our methodology, and the training thereof. The experimental results are given in Section 5, while the conclusions drawn from this work and suggestions for future work may be found in Section 7.