Geographically and temporally weighted neural network for winter wheat yield prediction

Highlights

• The existence of spatial and temporal non-stationarity was statistically tested.

• A GTWNN model was developed for winter wheat yield prediction in US.

• GTWNN outperformed three widely used models including GTWR, ANN and SVR.

• Moran's I values showed GTWNN's ability to address spatiotemporal non-stationarity.

Abstract

Accurate prediction of crop yield is essential for agricultural trading, market risk management and food security. Although various statistical models and machine learning models have been developed to enhance prediction accuracy, spatial and temporal non-stationarity, an intrinsic attribute of many geographical processes, is still rarely considered in crop yield modeling. From a statistical point of view, this study respectively provided evidence for the existence of spatial non-stationarity and temporal non-stationarity in winter wheat yield prediction based on geographically weighted regression (GWR) and temporally weighted regression (TWR). Then, a geographically and temporally weighted neural network (GTWNN) model was proposed by integrating artificial neural network (ANN) into geographically and temporally weighted regression (GTWR) using publicly available data sources, including satellite imagery and climate data. For a more credible evaluation, the leave-one-year-out strategy was adopted to make out-of-sample prediction resulting in a total of 12 test years from 2008 to 2019. The experiment results showed that the proposed GTWNN outperformed ANN, GTWR and support vector regression (SVR) achieving the average coefficient of determination (R²) values of 0.766, 0.759 and 0.720 at the three prediction times of end of July, end of June and end of May. Moreover, an extended Moran's I was adopted to assess the degree of spatiotemporal autocorrelation of the prediction errors. The error aggregation of GTWNN was lower than other models, indicating that GTWNN is applicable to addressing spatial non-stationarity in modeling the relationship between predictors and yield response. The methodology proposed in this paper can be extended to handle spatiotemporal non-stationarity in other crop yield predictions and even other environmental phenomena.

Introduction

Wheat is one of the earliest cultivated crop in the history of agriculture (Bell, 1987), and it continues to be vital in providing calories and nutrients for human and livestock today (Shewry and Hey, 2015). In addition to diet and health, wheat is even associated with general economic and social conditions (Curtis and Halford, 2014). As a major wheat producer and explorer, the United States (US) accounted for 7% of the world's wheat production and 14% of the world's wheat exports in 2019 (USDA, 2020a). According to the growing season, wheat is primarily classified into winter wheat and spring wheat. Winter wheat, planted in fall and harvested in the next summer, represents around 70% of total wheat US production with 35 million metric tons of harvest from 24 million acres in 2019 (USDA, 2020b). Timely, accurate and spatially specific winter wheat yield estimation therefore is important for food security monitoring and marketing planning on the local, national, and international levels (Cai et al., 2019).

The availability of growth conditions during the growing season has made it possible for researchers to perform accurate yield estimates. The low cost and wide spatial coverage of satellite imagery has also made up for the lack of ground observations, so that the scale of yield estimation has been expanded from field to regional, national and even global scales. Several studies have combined satellite imagery and statistical approaches to perform crop yield estimation (Balaghi et al., 2008; Bolton and Friedl, 2013; Kern et al., 2018; Ma et al., 2021). For example, Maselli and Rembold (2001) built linear regression models with monthly normalized difference vegetation index (NDVI) derived from the Advanced Very High Resolution Radiometer (AVHRR) images, to forecast cereal crop yield in four North African Countries. Becker-Reshef et al. (2010) developed a winter wheat yield prediction approach with time series NDVI derived from the Moderate resolution Imaging Spectro-radiometer (MODIS) data, and the regression model achieved satisfactory performance with an error less than 10%. Similarly, NDVI derived from MODIS was also adopted to predict winter wheat yield in Shandong, China using a linear model (Ren et al., 2008). Despite moderate success, the models established in these studies are global regression models, which means that the relationship between yield and explanatory variables is assumed to be spatially constant. However, in fact the relationship can be highly variable over space especially in large areas, because it is always affected by natural environment such as topography and soil type, and human factors such as farming habits and management system.

The phenomenon that an explanatory variable has different effects on model output over space is known as spatial non-stationarity or spatial heterogeneity (Stewart Fotheringham et al., 1996). Non-stationarity has been recognized by many researchers and has been taken into account in different areas such as environmental analysis and epidemiological spreading (Brunton et al., 2017; Lin and Wen, 2011; Song et al., 2014; Zhai et al., 2018). Moreover, the assumption of non-stationarity has been demonstrated to be of great significance to the improvement of model accuracy in the research of agricultural yield estimation (Shen et al., 2018). Therefore, various local modeling approaches have been developed to deal with the spatial non-stationarity issue (Huang et al., 2013; Imran et al., 2013; Shiu and Chuang, 2019). A commonly used approach in large-scale yield estimation is partition modeling, which assumes that in each partition, the relationship between dependent and independent variables is spatially stationary, and does not follow a location-specific regression function. Therefore, in partition modeling, a number of global models are developed independently for the geographic partitions. For example, to achieve better modeling accuracy, Manjunath et al. (2002) developed a linear yield prediction model for each of the 15 wheat-growing regions in Rajasthan, India, by integrating spectral and meteorological data. Mishra et al. (2008) established multiple district-level linear regression models to make in season yield prediction of sorghum in Burkina Faso, a West African country. However, the separation of limited datasets brings a tradeoff between achieving generalization performance and modeling spatial non-stationarity. Another model designed to address the issue of spatial non-stationarity is geographically weighted regression (GWR), which has been widely applied in many fields, such as geography and ecology, and has been demonstrated to be effective for crop yield prediction in agricultural research. For instance, Imran et al. (2015) compared GWR with conditional autoregression, a global spatial regression, when simulating the yields of sorghum, millet and cotton in West Africa, and found that the average accuracy improvement of GWR in semiarid zone and subhumid zone was 0.1 and 0.3, respectively. Shiu and Chuang (2019) who compared GWR with least squares ordinary in paddy rice yield estimation also observed similar results. Despite success, GWR has limitations in constructing spatial weight matrix due to the heavy reliance on the predefined spatial kernel functions (e.g. Gaussian, Bi-square, Exponential functions, etc). Essentially, spatial kernel functions are predefined distance decay functions in which larger weights are assigned to closer observations based on the Tobler's First Law of Geography (Tobler, 1970), and they are the simplified and idealized expressions of spatial proximity, making it possible to quantify the spatial relationships in modeling process. However, these kernels are relatively simple in addressing complicated interactions inherent in geographical relationships (Du et al., 2020). Additionally, although the selection of optimal bandwidth requires data-driven approaches such as Akaike Information Criterion (AIC), the predefined kernels expressed in formulas may hinder the precise quantification of spatial relationships using the inherent information in the data (Hagenauer and Helbich, 2021).

In addition to space, time is also an important dimension that affects the relationship between predictors and response (Fotheringham et al., 2015). Extensive efforts have been made to incorporate temporal effects into spatial regression. Among these, geographically and temporally weighted regression (GTWR) is one of the successful attempts for modeling non-stationary spatiotemporal relationships. It was originally achieved by redefining the distances between observations in the construction of the weight matrix. For example, Huang et al. (2010) formed a spatiotemporal distance through the linear combination of spatial and temporal distance, which was then improved by adding the interaction term of time and space dimensions on the basis of the original distance formula (Wu et al., 2014). Instead of improving the expression of spatiotemporal distance, Fotheringham et al. (2015) realized the expansion of GWR to GTWR by constructing a spatiotemporal weighting function which was expressed by the product of a temporal kernel function with a unique temporal bandwidth, and a spatial kernel function with a set of segregated spatial bandwidths over time. The various structures of GTWR adapt the interpretability of GWR, that is, they can generate a set of coefficients of the independent variables corresponding to a specific spatiotemporal coordinates, and output bandwidth which can be interpreted as spatiotemporal effects. In agriculture, factors including soil fertility changes and farming technology innovations are all possible causes of temporal non-stationarity. However, although GTWR has been successfully applied in various fields (Guo et al., 2017; Huang et al., 2010), it has never been used for crop yield estimation.

In contrast to conventional statistical methods, machine learning allows the appliance of complex mathematical calculations to large dataset, and it has demonstrated its superior ability in numerous fields such as natural language processing and image recognition (Goldberg, 2016; Nadkarni et al., 2011; Svyrydov et al., 2018). Increasingly, machine learning has been used in agriculture, such as crop yield estimation. For instance, Matsumura et al. (2015) compared the performances between artificial neural network (ANN) and multiple linear regression in predicting maize yield in Jilin province, China, and the results showed that ANN significantly outperformed the linear one. Sakamoto (2020) built a random forest (RF) regression algorithm with time-series MODIS satellite data and environmental records for soybean and corn yield prediction in the US, and the approach accurately predicted the yields of corn and soybean with root mean square error (RMSE) of 0.539 t/ha 0.206 t/ha, respectively. In addition, several researchers also have tried to establish machine learning based models considering spatiotemporal non-stationarity. For example, Crane-Droesch (2018) developed a semiparametric ANN using weather, soil, geographic coordinates and year data to predict corn yield across multiple geographically contiguous states in the US. Shook et al. (2018) built a Long Short Term Memory based framework with weather and genotype variables for soybean yield estimation in North America, and the model achieved improved performance by incorporating GPS coordinates and predicting year. Similar ideas can be found from these studies that spatial and temporal information are added as independent and parallel factors with other variables into the machine learning method. However, in fact, each location in spatiotemporal dimension is characterized by a series of physical and human properties, and it is these properties, not the location, that directly affect the growth and yield of crops. Moreover, the machine learning model performance can potentially be further enhanced by incorporating prior knowledge of the crop.

In this study, we proposed a geographically and temporally weighted neural network (GTWNN) considering spatial and temporal non-stationarity for county-level winter wheat yield estimation in the US. Before modeling, we first performed statistical tests to provide evidence for the existence of spatial and temporal non-stationarity in winter wheat yield estimation based on GWR and temporally weighted regression (TWR) respectively. A GTWNN model was then developed with the smoothed sequential features extracted from satellite images and meteorological datasets from 2008 to 2019. Furthermore, the proposed model was compared with GTWR, ANN and support vector regression (SVR) for performance evaluation. Finally, the spatiotemporal patterns of the prediction errors derived from the four models were analyzed.