1 Introduction

Recent analysis of Australia’s wheat yields (Hochman et al. 2016) has shown that over the 17 years from 1996 to 2012 the average Australian annual yield of 1.73 t/ha was 50% of the average water limited yield of 3.45 t/ha. Allowing for diminishing returns and logistical realities, the exploitable water-limited yield is often set at 80% of the water-limited yield potential (Lobell et al. 2009; van Rees et al. 2014). For wheat, this still leaves an exploitable yield gap of 1.03 t/ha, i.e. the difference between the attainable yield and the actual yield most farmers achieve (van Ittersum et al. 2013). With an average national wheat production of 21.5 Mt, the national exploitable yield gap was 12.7 Mt. The annual value of the exploitable yield gap for wheat (valued at 250 AUD/t) was 3184 million AUD.

While Australian wheat producers are closing the yield gap at a rate of about 1% per annum, actual wheat yields have stagnated since 1990 (Hochman et al. 2017). On-going increases in the productivity of the grains industry will be difficult to achieve until the causes of the unrealised yield potential are identified and quantified so that appropriate (financially viable) management solutions can be identified and implemented by farmers. In a related study, Hochman and Horan (2018) conducted in silico experiments over 15 years at 50 weather stations to ascertain the impact on grain yield of suboptimal practices against the ‘best management practice’ rules that were used to calculate the benchmark water-limited yields. In terms of the proportion of the yield gap, average national losses per suboptimal practice were the average N fertiliser application rate (45 kg N/ha/crop), 40%; conventional tillage, 33%; suboptimal weed control during the summer fallow, 26%; low seedling density, 12%; and a two-week delay in sowing, 7%. Other factors that contribute to the yield gap include biotic stresses such as plant diseases, insects and other pests, in-crop weeds and extreme weather events (e.g. floods, strong winds and hail). They also investigated the opportunity to lift the water-limited yield by adopting an emergent new management practice of sowing on an optimised site-specific date that is earlier than the traditional sowing window described for the currently accepted best practice (Flohr et al. 2017). This emergent practice, matched with slower maturing varieties and additional N inputs as required, was found to have the potential to increase wheat yields nationally by 30%.

To gain insight into why yield-optimising practices are not more widely adopted, 232 farmers from 14 grain-growing statistical local areas (SA2s) in seven GRDCFootnote 1 subregions were interviewed (Zhang et al. 2019) (Fig. 1). The data collected revealed significant differences between farms with smaller yield gaps and those with greater yield gaps in relation to crop management practices, farm characteristics and socio-psychological characteristics of farm managers. The study found that farms with smaller yield gaps are likely to be smaller holdings growing less wheat on more favourable soil types, are more likely to apply more N fertiliser, to have a greater crop diversity, to soil test a greater proportion of their fields, to have fewer resistant weeds, to adopt new technologies and to be less likely to grow wheat following either cereal crops or a pasture. They are more likely to use and trust a fee-for-service agronomist and to have a university education. This study demonstrated that yield gaps are the result of the intertwined dynamics between biophysical factors, socio-psychological characteristics and farm management practices. Socio-psychological factors not only directly contribute to yield gap but also influence farm management practices that in turn contribute to yield gaps. These findings align with the broader realisation that, to close yield gaps, it is important to develop integrated strategies that address both farm management dimensions and the complexity of the decision-making process (e.g. Antle 1987; Bowman and Zilberman 2013; Giller et al. 2011; Hardaker et al. 2015; Van Dijk et al. 2017; Van Rees et al. 2014).

Fig. 1
figure 1

a Wheat and canola fields on a South Australian farm. b Locations of surveyed local statistical areas (SA2s) with contrasting average relative yields. The relative yield of wheat (% of water-limited yield) is indicated by the red-yellow colour gradient. The white borders show the GRDC subregions of the Australian grain zone.

Full size image

We adopt the integrated framework of van Dijk et al. (2017) as a starting point for disentangling agronomic and economic yield gaps. Their approach integrates agronomic definitions of the yield gap (Evans and Fischer 1999; van Ittersum et al. 2013) into an efficiency frontier framework (Coelli et al. 2005). This framework decomposes the agronomically defined yield gap into four categories: (1) the technical efficiency yield gap which describes the distance from the production frontier at a given level of inputs, (2) the allocative yield gap which describes the gap due to insufficient inputs to maximise profit at the technical frontier, (3) the economic yield gap which is the gap between yields achieved with unlimited resources and the yield achieved at the technical frontier when inputs are reduced to maximise profit and (4) the technology yield gap which is the gap between ‘feasible yields’ and the potential (water-limited) yields which can only be achieved using advanced technologies and the latest varieties (Van Dijk et al. 2017).

While embracing the integrated framework, we found it incomplete for the purposes of describing wheat yield gaps in an advanced economy such as Australia. Two amendments were required to enable a more nuanced discussion about the agronomic and economic aspects of the yield gap. First, given Australia’s highly variable climate, much of the allocative yield gap is due to risk aversion while the lack of credit markets, high transaction costs and information asymmetries are less important than in developing countries. We therefore find it helpful to divide the allocative yield gap into two categories: (1) a resource-constrained yield gap and (2) a risk aversion yield gap. Second, surveys of Australian growers’ yield gaps (van Ittersum et al. 2013, Zhang et al. 2019) show that in a given season 10–20% of fields exceeded the simulated water-limited yield. This implies that the technology yield gap described by van Dijk et al. (2017) is only part of the story. With technology moving rapidly, innovative growers’ practices may be more efficient than the accepted ‘best management practice’ which is part of the definition of water-limited yield potential. An example of such an emerging technology in Australia is early sowing with longer season varieties which has the potential to increase yields by 30–40% with little change in input levels (Hochman and Horan 2018; Hunt et al. 2019). We therefore consider an additional ‘emerging technology’ efficiency frontier in which technical efficiency is higher than the currently accepted water-limited potential for any given level of inputs.

In the yield gap efficiency frontier framework offered here (Fig. 2), average yields (Ya) for a given environment and its range of seasonal conditions are achieved by growers applying average inputs (IA) and common practices, average technologies and currently popular varieties with average agronomic efficiency. Water-limited yields (Yw) can be achieved when enough resource inputs are applied to maximise yields and when best practice and current technology, including current genetics, are deployed.

Fig. 2.
figure 2

Yield gaps in an efficiency frontier framework. The dash dot dash (green) line represents the relationship between inputs and yields for a grower with an average agronomic efficiency; the solid (red) line represents relationship between inputs and yields for a grower with best management practice using current technology; the dashed (blue) line represents an emerging frontier using innovative practices and emerging technologies. Input levels range from the local average (IA) to the level of inputs where maximum yield can be achieved (IMAX), with IP representing the level of input at which profit is maximised and ICE representing the optimal level of input allowing for a grower’s risk aversion (certainty equivalents).

Full size image

Considering the diminishing rates of returns to additional resources, economically rational growers may apply less inputs (IP) and reduce yields to maximise profits. A risk averse grower may reduce inputs further (ICE) by sacrificing profits in good seasons to reduce their downside risk in poor seasons. A resource-constrained grower may not be able to afford or to borrow more than the amount of inputs signified by (IA). The yield implications of these three input levels depend on the growers’ agronomic efficiency. By merely increasing inputs without a change in management efficiency, the grower realising A will move along a path signified by positions B and C. However, by using the same level of inputs more efficiently, the grower at A can move to position D, while a grower at D can move to positions E, F or Yw.

Because technology is moving rapidly, and innovative growers’ practices may be more efficient than the accepted ‘best management practice’, we also consider a third efficiency frontier in which growers move from G to H to I and to Ymax along a line that describes the emerging agronomic efficiency frontier (or the leading edge). The difference between Ymax and Yw is the innovation yield gap. Given the diversity of growers, we can expect to find growers’ fields distributed throughout the area below the dashed (blue) line from the origin to Ymax.

With this framework in mind, we can more clearly distinguish between agronomic and economic yield gaps. While the total yield gap is the difference between Yw and Ya, the yield difference along the path from Yw to D (via F and E) may be described as the socio-economic yield gap as it is constrained by the cost of inputs. This socio-economic gap can be further differentiated between yield lost due to restricting inputs to maximise profitability (rational economic yield gap), yield lost due to risk aversion and yield lost due to lack of access to financial resources.

The gap between D and A, E and B and F and C describes the agronomic yield gap at various economically determined input levels. The agronomic yield gap arises from management inefficiencies in timing of operations and in minimising the impacts of biophysical stresses, both biotic (such as weeds, pests and diseases) and abiotic (such as water and nutrient deficits) that lead to yield reductions. When inputs are less than IMAX, agronomic efficiency requires a good balance of limited resources. For example, should the grower spare fertiliser to ensure that fallow weeds are fully controlled? Such decisions require good agronomic knowledge which can be improved by spending resources on agronomic consultants and/or self-education. Thus, agronomic efficiency is also dependent on psycho-social factors (Zhang et al. 2019). By using simulation to calculate yields at various input levels, we remove the agronomic yield gap from our calculations to enable us to focus on the socio-economic yield gap or the difference between Yw and Yse.

In the current study, we apply a profit-risk-utility framework to the 14 SA2s from the Zhang et al. (2019) study to better understand the socio-economic constraints to closing these yield gaps. In particular, we investigate the effect of the farmer risk attitude on the choice of agronomic practices, grain yield and the yield gap across agroclimatic zones of the Australian wheatbelt. The intention here is to use this analysis to gain an understanding that will be used to inform agricultural consultants to more effectively guide their grain grower clients through a process of closing the yield gap while allowing them to work within their clients’ aversion to risk. We also contribute to more general insights on the determinants of yield gap globally and the role of integrated modelling in yield gap analysis.

2 Materials and methods

2.1 Case study sites

This study focuses on 14 contrasting local areas (SA2s; roughly equivalent to a shire) in the Australian grain zone (Fig. 1). The sites are grouped according to their average annual water-limited yield (Yw)—low, medium and high—as described in Hochman and Horan (2018). A summary of key data characteristics for each site is provided in Table 1. These include climatic and economic information, such as production costs per effective hectare as well as SA2 level data on the water-limited yield (Yw) and the average yields achieved by farmers in their SA2s (Ya).

Table 1 Site climatic and economic information, including production costs per effective hectare. 1SA, South Australia; Vic, Victoria; WA, Western Australia; Qld, Queensland; NSW, New South Wales. 2Daily climate data from the SILO historical climate database for each site. 3Actual farmer yields extracted from www.yieldgapaustralia.com.au. 4Production costs include a mix of variable costs such as fertilisers other than N, herbicides applied in-season, fuel and oil and fixed costs apportioned on a AUD/ha basis (e.g. repairs and maintenance, labour, insurance and levies), interest on variable costs (8%) and depreciation of machinery investment (10% of 200 AUD/ha in machinery investment)
Full size table

2.2 Scenario analysis

2.2.1 Simulating crop yield

The Agricultural Production Systems Simulator (APSIM v.7.8) (Holzworth et al. 2014) was used to model water-limited wheat grain yield over the 2001 to 2015 growing seasons using the climate files described in Table 1. Rules used in APSIM to produce water-limited yield and other agronomic scenarios include sowing rules, N fertiliser rules and soil initialisation and annual reset rules (see Section 2.2.2 for details). In the current study, best management practice rules (e.g. sowing rules and N fertiliser rules) are those used in Hochman and Horan (2018) and readers are referred there for more details and justification of these rules.

The use of APSIM for the simulation of wheat and soil water and nitrogen response to different seasons and nitrogen management strategies has been widely tested and validated in Australian cropping systems (e.g. Hochman et al. 2016; Monjardino et al. 2015; Sadras and Rodriguez 2010; Van Rees et al. 2014).

2.2.2 Simulating agronomic scenarios

Water-limited yield

The water-limited yield (Yw) is the benchmark treatment in this analysis. Yw represents the yield that can be achieved by rainfed crops when grown with best management practices under current technology, with nutrients non-limiting and biotic stress effectively controlled. Under conditions that can achieve Yw, crop growth rate is determined only by available water, solar radiation, temperature, atmospheric CO2 and genetic traits that govern length of growing period (cultivar maturity) and light interception by the crop canopy (e.g. canopy architecture). Yw is location specific because of the climate and soil properties that govern soil water availability based on available water storage capacity, rooting depth and soil constraints such as salinity or physical barriers to root proliferation (Van Ittersum et al. 2013).

Sowing rules

All Queensland and New South Wales (NSW) sites north of latitude 32.24 (Dubbo) were classed as northern sites and used the northern sowing rule; all other sites used the southern sowing rule:

  • Northern sites: sow if rain ≥ 15 mm over 3 days and PAW (plant available water) ≥ 30 mm (from 26 April–15 July).

  • Southern sites: sow if rain ≥ 15 mm over 3 days regardless of soil moisture (from 26 April–15 July).

In both cases, the crop is sown on 15 July if criteria are not met during sowing window. Other key sowing rules include sowing density = 150 plants/m2, row spacing = 250 mm and sowing depth = 30 mm.

N fertiliser rules

At sowing, add 100 kg/ha NO3 minus soil nitrate in top 60 cm of soil on April 26Footnote 2. Check top 60 cm soil daily, if NO3 < 80 kg/ha and PAW ≥ 30 mm and Zadoks growth stageFootnote 3 ≥ 10 and ≤ 49 then add 70 kg N/ha (max 1 application).

Soil initialisation and annual reset rules

Because initial soil moisture is an important but unmeasured parameter at the start of the simulation period of interest, initial soil water is arbitrarily set to 10% of plant available water capacity (PAWC) 15 years ahead of the start date of the simulation in order to allow soil water to find its correct level at the start of the simulation period (the first 15 years’ data are discarded). Soil organic carbon is initiated as per soil profile data. Initial soil NO3 is set to 25 kg/ha for each metre depth of soil; initial soil NH4 is set to 5 kg/ha for each metre depth of soil. Initial surface organic matter is set to 100 kg/ha with the C:N ratio set at 80. Surface organic matter is not reset; soil water is not reset; soil NO3 and NH4 are reset at crop maturity except for N rate-dependent treatments where they are not reset.

Agronomic treatments

The agronomic treatments simulated for this study are listed below. These include a nominal site practice treatment that is designed to mimic site-dependent ‘typical practices’ with regard to multiple practices: N application rates (these are differentiated according to a site’s classification as having low, medium or high Yw), plant sowing density, timely sowing and controlling fallow weeds. Site practice corresponds to point D in Fig. 2. Other treatments, such as N45Split2, Fallow Mgt6, SowDelay and Plants75 are treatments where individual elements of the Yw best management practice are set at input levels that are between IA and IMAX.

At the other end of the spectrum, Sow26April and OptTOS+Var are emerging practices that have the potential to lift the yield frontier to Ymax. Compared to the Yw treatment, the full potential of the emerging practice achieved by the Ymax treatment requires unlimited N in addition to optimised time of sowing and cultivar (Hochman and Horan 2018). The 15 agronomic treatments are described next, including the rules used in their simulation.

Yw Water-limited yield

Site Practice A combination of N45Split2, Plants100, and FallowMgt6 (explained below) with a two-week delay in sowing

N45Split1 N rate dependent on whether average annual Yw is low, medium or high

  • If Yw ≤ 2.4 t/ha apply 22.5 kg N/ha at sowing

  • If Yw > 2.4 t/ha and Yw ≤ 3.5 t/ha, then apply 45 kg N/ha at sowing

  • If Yw > 3.5 t/ha apply 67.5 kg N/ha at sowing

N45Split2 N rate dependent on whether average annual Yw is low, medium or high

  • If Yw ≤ 2.4 t/ha apply 30 kg N/ha at sowing

  • If Yw > 2.4 t/ha and Yw ≤ 3.5 t/ha, then apply 45 kg N/ha at sowing

  • If Yw > 3.5 t/ha apply 60 kg N/ha at sowing

N90Split N rate dependent on whether average annual Yw is low, medium or high

  • If Yw ≤ 2.4 t/ha apply 45 kg N/ha at sowing

  • If Yw > 2.4 t/ha and Yw ≤ 3.5 t/ha, then apply 90 kg N/ha at sowing

  • If Yw > 3.5 t/ha apply 135 kg N/ha at sowing

FallowMgt5 Summer fallow weeds are sprayed out 2 weeks after rainfall event (10 mm in 3 days). This gives the number of times the fallow is sprayed for Yw (best practice)

FallowMgt6 Summer fallow weeds are sprayed out 6 weeks after rainfall event (10 mm in 3 days)

Plants50 Sowing density of 50 plants/m2

Plants75 Sowing density of 75 plants/m2

Plants100 Sowing density of 100 plants/m2

Plants125 Sowing density of 125 plants/m2

Sow26April Sow on 26 April every year

SowDelaySowing delay due to conventional tillage requiring ≥ 25 mm rain over 3 days instead of ≥ 15 mm

OptTOS+VarSow on highest yielding sowing date selected from simulations of crops sown every 7 days from 5 April to 21 June using the cultivar that was highest yielding on average over 15 years (from current varieties representing early, mid-early, mid, mid-late and late maturity types)

YmaxOptTOS+Var analysis was redone with additional N applications to ensure the time of sowing by variety combinations were not N limited

2.2.3 Yield gap indicators

The main yield measure of the current yield gap used in the analysis is the relative yield (Y%) based on 15-year average wheat yields. The relative yield of any treatment is its yield as a percentage of the water-limited yield (Lobell et al. 2009), expressed as

$$ Y\left(\%\right)=\frac{100\times Y}{Yw} $$
(1)

In addition, we calculate the following yield gaps to contextualise the analysis in terms of the yield gap situation in Australia, where Ya is actual farmer yields extracted from www.yieldgapaustralia.com.au:

  • Agronomic efficiency yield gap (AEYg) = site practice–Ya

  • Socio-economic yield gap (SEYg) = Yw–site practice

  • Innovation yield gap (IYg) = Ymax–Yw

2.3 Profit-risk-utility framework

For the purposes of this study, we simplified the approach of Monjardino et al. (2013, 2015) to profit, risk and utility, based on expert feedback and new applications (e.g. Komarek et al. 2018). We named this simplified approach to calculating profit, incorporating risk and prioritising utility the profit-risk-utility framework (PRUF). The key components of PRUF are described next.

2.3.1 Calculating profit

Mean net return

The mean net return of producing a wheat crop in each agronomic scenario was calculated via a profit function modified from Monjardino et al. (2013, 2015) in order to accommodate for treatment variables other than N rates, such as crop density and summer weed control:

$$ NR=\left(Y\times Pw\right)-\left(\left(N1+N2\right)\times Pn\right)-\left( Cd\times d\right)-\left( Cs\times s\right)-\left( Cw\times h\right)- Co $$
(2)

Where NR is the mean net return (AUD/ha); Y is the yield of the wheat crop (kg/ha); Pw is the price of Australian Standard White (ASW) wheat grain (AUD/kg); N1 is the rate of N applied at sowing (kg N/ha); N2 is the rate of N applied in season (kg N/ha); Pn is the price of N (AUD kg/ha of N; i.e. price of urea/0.46); Cd is the operational cost of top-dressing with N fertiliser in-season (AUD/ha); d is the number of N top-dressing applications; Cs is the cost of seeding (seed price + seed treatment) (AUD/kg); s is the seeding rate (kg/ha); Cw is the cost per application of summer weed control (AUD/ha); h is the average number of herbicide sprays; and Co are the other costs (AUD/ha).

The operational cost of top-dressing with N fertiliser in-season is determined by site annual average Yw (10, 15 and 20 AUD/ha for low, medium and high Yw area, respectively) to reflect potential variation in fuel and oil costs associated with heavier/wetter soils. The cost of seeding assumed in the calculations is 0.35 AUD/kg of seed (price of seed at 0.3 AUD/kg of seed + seed treatment at 0.05 AUD/kg of seed). The seeding rate is calculated for each plant density level using standard wheat seed information (germination at 95% and seed weight at 4.5 gm/100 seeds). The other costs, assumed unchanged in this short-run analysis, include input costs of growing the wheat crop (e.g. fertilisers other than N, herbicides applied in-season; casual labour, fuel and oil required for the combined basic operation of seeding, spraying and applying fertilisers upfront including N), fixed costs of production apportioned on an AUD/ha basis (e.g. labour, repairs and maintenance, insurance and levies), interest on variable costs (at an assumed rate of 8%) and depreciation of machinery investment (assumed 10% of average 200 AUD/ha in machinery investment). Some of these costs, such as in-season herbicide sprays, are assumed to vary with site annual average Yw. A summary of all production costs for each site is shown in Table 1.

Datasets required for the profit function included two price series, one for ASW wheat and the other for N fertiliser (urea, 46% N), along with key variable and fixed costs were obtained from a range of data sources including commodity statistics (ABARES 2014) and farm budget guides (DAFWA 2015; Rural Solutions SA 2011). Real prices at farm gate were used to capture long-term deflation over the 15 years from 2001 to 2015 (adjusted to 2001 base year, using the consumer price index). For each price series, we calculated the mean price of wheat (310 AUD/tonne) and N (823 AUD/tonne) over that period.

Key economic indicators

The three key economic indicators used in the analysis are

  • Mean of expected net return (NR)—i.e. the expected magnitude of economic net return or risk-neutral profit

  • Standard deviation of net return (SD)—i.e. a measure of variance or dispersion from the mean

  • Coefficient of variation (CV)—i.e. a measure of dispersion of a probability distribution (SD/mean)

2.3.2 Incorporating risk

Dryland farming is risky. Yield and price risk are the main contributors to financial risk in low-medium rainfall environments (Bowman and Zilberman 2013; Kingwell 2011).

Yield risk

PRUF captures yield risk, or yield variability, through the APSIM generation of frequency distributions of wheat yields for each of the agronomic scenarios considered in the study. Using the @RISKTM software (Palisade Corporation 2002), the yield frequency distributions were fitted using probability density functions (PDF) of various forms including Lognorm, InvGauss, ExtValueMin, Weibull, Pearson5, Normal, Loglogistic, Uniform and Beta distributions. As described by Monjardino et al. (2013, 2015), we chose the Anderson-Darling (AD) statistics test to measure the goodness of fit of each distribution. The PDF with the best fit as measured by the AD statistic test (first row for each site in Table 2) was selected for use in Monte Carlo simulation of net economic returns through the process of generating 1000 random iterations to sample from the probability distribution.

Table 2 Yield PDF value and economic risk measures for the 15 agronomic treatments applied to a wheat crop for Loxton (SA), Northam (WA) and Quirindi (NSW). The agronomy efficiency yield gap, socio-economic yield gap and innovation yield gap, as well as the risk-adjusted profit values are illustrated in Figs. 3 and 4.
Full size table

Price risk

A similar process described for yield risk was applied to price risk as a means to incorporate long-term price volatility. Frequency distributions of 15-year real prices for wheat grain and fertiliser N were best fitted using various PDFs. Wheat prices were found to be normally distributed over the 2001–2015 period, whereas N prices best fitted a Laplace distribution, reflecting the large price spike that occurred in the late noughties (2007–2009). No significant correlation was found between both prices, so these price distributions were used in calculating economic net returns from growing wheat under a range of agronomic practices.

Financial risk

In this study, the quantification of financial risk involved two steps:

  1. 1.

    Capturing the variability in net returns for each scenario by using @RISK to generate random simulations of net returns (using Eq. 2) with random samples for both the yield parameter Y and the price parameters Pw and Pn, drawn from the modelled PDFs for yields and prices over the defined period. Like for yield and prices, PDFs were fitted to frequency distributions of net returns for each scenario, and the best PDF was selected using goodness of fit and AD test.

  2. 2.

    Calculating two key indicators of financial risk to use in the analysis:

    • Probability of break-even (P (NR ≥ 0)), i.e. the probability of returning a profit or positive mean net return

    • Conditional value at risk of the lowest 10% of possible outcomes (CVaR0.1), i.e. the mean of the lowest 10% net returns or, in other words, the risk of extreme financial loss associated with unfavourable events

2.3.3 Prioritising utility

Farmers do not seek to maximise yield and rather seek to maximise expected profit. But farmers are also often averse to risk, especially those operating in dryland environments who are more exposed to financial loss, or downside risk, from unfavourable weather events or market conditions (Hardaker et al. 2015). This means that for a more realistic economic analysis, profits should be adjusted for risk. However, farmers with different levels of risk aversion are likely to have different preferences for management strategies with varying risk profiles, so a range of attitudes to risk should be considered.

Risk premium and certainty equivalent

Prioritising risk aversion means that a farmer’s objective shifts from maximising expected profit (i.e. risk-neutral profit) to maximising expected utility, or certainty or overall satisfaction (Hardaker et al. 2004). In other words, risky strategies are re-ranked and compared based on their level of certainty or preference for a farmer with a particular risk attitude.

In this study, we employ a simplified approach of that described in Monjardino et al. (2013, 2015), whereby N fertilization preferences under risk were ranked through a stochastic efficiency with respect to a function (SERF) analysis (Hardaker et al. 2004) via a measure of certainty equivalent (CE), or risk aversion, calculated through a utility function. While the SERF approach meets the needs of this study, it is theoretically difficult to apply the utility function to a partial analysis (e.g. crop net returns) and not whole-farm returns and even more difficult to estimate what levels of attitudinal risk aversion farmers hold.

The simplification in the approach used here concerns the method by which CE is calculated. While in SERF the certainty equivalent is determined under a utility function of a decision maker with wealth (based on mean net returns) as the performance criterion within limits of a coefficient of risk aversion scale, in this study, we calculate CE as the difference between the expected mean net return and a risk premium (RP), i.e. CE = NR–RP (Antle 1987; Chavas and Shi 2015; DiFalco et al. 2007; Komarek et al. 2018; Lehmann et al. 2013). RP is best defined as ‘the smallest amount of certain money a decision maker is willing to pay to eliminate risk exposure’ (DiFalco et al. 2007). RP captures the cost of risk measured through mean, variance and skewness of NR distributions and can be approximated as

$$ RP=0.5\times \frac{r}{NR}\times V $$
(3)

where r is a coefficient of relative risk aversion, NR is the mean net return and V is the variance of the mean net return for each agronomic scenario (e.g. Komarek et al. 2018; Lehmann et al. 2013).

The risk attitude range is typically measured by a risk aversion coefficient, measuring either absolute or relative risk aversion, based on the magnitude and spread of the distribution of net returns (e.g. Hardaker et al. 2004) and the actual risk averseness of decision makers (Grové and Oosthuizen 2010). Since farmers tend to have decreasing absolute risk aversion at higher wealth, a constant relative risk aversion coefficient was deemed more suitable to this analysis. The values of 0–4 for the coefficient of relative risk aversion used in this study were selected on the basis of r range estimates proposed in published econometric studies (Gandrofer et al. 2011). Therefore, we assume that 0 = no risk aversion (i.e. risk-neutral decision maker), 1 = low risk aversion, 2 = moderate risk aversion, 3 = high risk aversion and 4 = very high risk aversion. The analysis can be run for each of these r values across the full spectrum of attitudes to risk, or the user can select the relevant level, or r, if known.

Risk-adjusted profit and maximum cost of risk aversion

The key indicators of farmer risk aversion are the risk premium and the associated certainty equivalent (i.e. the difference between net return and risk premium) for each scenario. However, we simplify the analysis further by referring to the certainly equivalent as risk-adjusted profit (RAP). A different RAP value is calculated for each of the five levels of risk aversion (i.e. 0, 1, 2, 3, 4). When the risk aversion coefficient is nil, RP = 0 and CE = NR, so the result is risk-neutral profit. The difference between risk-neutral profit and risk-adjusted profit at the maximum level of risk aversion (r = 4) is the maximum cost of risk aversion.

3 Results and discussion

The magnitude and va iability of crop yield and net returns across the full range of agronomic scenarios for the 14 sites were assessed against the ten key indicators described in Section 2.3 and shown in Table 2, except for the yield gap indicators (2 to 4), which are illustrated in Fig. 3 because they translate into a single value per site. Regarding indicator 10, the maximum cost of risk aversion is included in Table 2, while the range of risk-adjusted profit values per scenario is shown in Fig. 4.

Fig. 3
figure 3

Average wheat yield (bars) and mean net return (black line) achieved by the simulated site practice (blue bar), Yw (orange bar), Ymax (green bar) and profit-maximising practice (Pmax) (purple bar) for a a low-yielding site (Loxton, SA), b a medium-yielding site (Northam, WA) and c a high-yielding site (Quirindi, NSW). The actual farmer yield (Ya) (grey bar), agronomic efficiency yield gap (aqua bar), socio-economic yield gap (pink bar), site (total) yield gap (striped aqua/pink bar) and the extra innovation yield gap (yellow bar) are shown as well for all three sites.

Full size image
Fig. 4
figure 4

Risk-neutral profit (grey bars) and associated risk-adjusted profit for the 15 agronomic practices across four levels of risk aversion (low, green bars; moderate, red bars; high, purple bars and very high, blue bars) for a a low-yielding site (Loxton, SA), b a medium-yielding site (Northam, WA) and c a high-yielding site (Quirindi, NSW). The practices are ranked by risk-neutral profit and the preferred agronomic practice for each level of risk aversion is outlined in black.

Full size image

The ten indicators considered are (1) relative yield, (2) agronomic efficiency yield gap, (3) socio-economic yield gap, (4) innovation yield gap, (5) mean of expected net return (i.e. risk-neutral profit), (6) standard deviation of net return, (7) coefficient of variation, (8) probability of break-even, (9) conditional value at risk of the lowest 10% of possible outcomes and (10) risk-adjusted profit/maximum cost of risk aversion.

3.1 Maximising yield

Table 2 shows the percentage of water-limited yield (the relative yield, Y%) achieved by the 15 agronomic practices for one example site in each Yw area—Loxton SA, Northam WA and Quirindi NSW. Across all sites, the yield achieved with the site practice varied between ~ 68 (e.g. Northam, Quirindi) and ~ 75% (e.g. Loxton) of Yw. There was a further opportunity to increase the yield frontier by between 19 (e.g. Loxton) and 31% (e.g. Quirindi) with novel practices resulting from site-specific adjustments to the Yw practice, such as with the OptTOS+Var and Ymax treatments.

Comparative yield gaps are illustrated in Fig. 3. Loxton had the lowest site yield gap (0.46 t/ha), 84% of which was attributed to socio-economic factors and only 16% to agronomic efficiency factors. The 1.89 t/ha yield gap calculated for Northam was more evenly divided between socio-economic (65%) and agronomic efficiency (35%) gaps. Quirindi recorded the highest total yield gap (2.28 t/ha) of which 73% was due to socio-economic drivers and 27% to agronomic efficiency.

Overall, the yield-maximising treatment involved a combination of higher N inputs (up to 150 kg N/ha), more intensive weed control in the fallow, a crop density of 150 plants/ha, a more suitable wheat variety (e.g. in OptTOS+Var) and an optimised site-specific time of sowing date that is earlier than the start of the traditional sowing window described for the currently accepted best practice for each site. As discussed by Hochman and Horan (2018), there is a clear opportunity to lift the water-limited yield frontier by adopting emergent new management practices, especially in higher yielding areas such as Quirindi, where innovation gaps of up to 1.63 t/ha were recorded (versus ~ 0.30 t/ha in lower yielding sites such as Loxton).

Practices that yielded less across the range of sites include delayed sowing (SowDelay), with 67–89% of Yw, which in the case of Northam was slightly lower than the site practice (68%). Less weed control in fallow resulted in a higher yield gap than when more herbicides were sprayed during the fallow (FallowMgt6) and lower crop densities (especially Plants50 and Plants75), as demonstrated for low, medium and high yield potential sites (Table 2).

The downside yield risk in modelled output is likely to be less than that occurring in the field. This is due to a number of factors including the inability of the model to accommodate the effects of pests, diseases, extreme weather events such as frosts and heat stress and limiting nutrients other than nitrogen and the chance that the more timely management of N application, fallow weed management, plant density, crop sowing and crop variety, may not necessarily be logistically achievable in the whole-farm context.

3.2 Maximising profit

When typical costs were built in to allow net returns to be calculated, the potential benefits from a range of strategic combinations of N fertilisation, in-fallow weed management, crop seeding density and time of sowing were evaluated for the 15 agronomic treatments considered in Section 2.2.2. For each site, we initially compared site practice with all other practices using the basic parameters of mean net return, standard deviation from the mean and coefficient of variation.

Overall, across all 14 sites, mean annual net returns varied between 476 AUD/ha (FallowMgt6) in the four low-yielding areas (Loxton, Buloke, Cunderdin, Le Hunte-Elliston) and 2317 AUD/ha (Ymax) in Quirindi (NSW). In most sites, the highest returns occurred with Ymax and/or OptTOS+Var. The lowest returns generally resulted from practices with low N and herbicide inputs and/or delayed sowing across all sites (site practice, FallowMgt6, SowDelay, N45Split1, N45Split2).

Poor fallow weed control and delayed sowing resulted in reductions in mean net returns and increases in the CV of the mean net return in most sites, especially in the low-yielding and medium-yielding sites (Table 2). This suggests that allowing weeds to thrive in the fallow, which contributes to a reduction in soil moisture and to delaying the sowing of the crop, clearly offsets the benefit of higher N inputs associated with these practices while increasing the risk of crop under-performance. These results and the interactions between N and fallow weeds were analysed in-depth by Hochman and Horan (2018).

Notably, we found that despite the Ymax treatment providing the highest water-limited yield in most sites (Fig. 3), the higher costs associated with the additional N inputs meant that maximum profit (Pmax) was sometimes (e.g. in Northam) achieved by the OptTOS+Var treatment with a lower N input and the optimised time of sowing by variety (Fig. 3). Overall and across all sites, both Ymax and OptTOS+Var consistently outperformed Yw as well as site practice in terms of yield and profit.

In reality, the difference between the yield realised by leading farmers who manage to remain economically viable (profit-maximising yield) and the yield achieved by the majority of farmers who make less profitable management decisions constrained by a range of agro-ecological and socio-economic factors (actual yield) is likely to be even more substantial than the gap obtained with a standard site practice treatment as a result of logistical limitations (e.g. farm size) (Fig. 3). This suggests a significant opportunity to close the socio-economic yield gap in Australia by developing robust and flexible crop management plans that adapt to seasonal and market volatility, capitalise on the favourable years and retract on the low-yield potential years.

3.3 Minimising downside risk

The financial risk of each treatment was assessed through the probability of break-even and the conditional value at risk of the lowest 10% of possible outcomes. The probability of breaking even was very high (93–100%) in most simulated scenarios, especially for the higher yielding sites as exemplified in Table 2. Downside risk using CVaR0.1 generated values up to − 97 AUD/ha (i.e. higher downside risk) at plant densities well below (Plants50, Plants75) those required to achieve yield potentials in the low-yielding sites. There were no negative CVaR0.1 values for all treatments and downside risk was low (i.e. higher CVaR0.1 values) in all the higher yielding sites, with the highest CVaR0.1 value of 1678 AUD/ha calculated for Ymax in West Coast (an improvement of 805 AUD/ha when compared with the site practice.

3.4 Maximising risk-adjusted profit

The results generated so far assume farmers’ risk-neutral behaviour where maximising mean net returns, or risk-neutral profit, is the objective regardless of variance. When accounting for four levels of risk aversion (low, moderate, high and very high), we found that maximum risk-adjusted profit was achieved across all risk aversion levels by the Ymax treatment in some of the medium and high Yw sites (Central Highlands, Goyder, St Arnaud, Gilbert Valley and Quirindi). Similarly, the OptTOS+Var treatment maximised risk-adjusted profit across all risk aversion levels in several low and medium Yw sites (Buloke, Le Hunte-Elliston, Claremont and Northam). However, at a number of sites across the Yw yield range (Loxton, Cunderdin, Karoonda-Lameroo, West Coast and Gunnedah), other treatments maximised the risk-adjusted profit for farmers highly averse to risk (particularly downside risk, as shown in Table 2 for Loxton), despite the Ymax treatment providing the highest water-limited yield potential and often also the highest risk-neutral profit (Fig. 4). This means that optimising the time of sowing by crop variety is a risk-reducing practice and could become the preferred treatment of more risk-averse farmers, regardless of yield potential, although the shift to OptTOS+Var only occurs at high to very high-risk aversion levels in the high-yielding sites (Gunnedah and West Coast).

As expected, farmers are more likely to forego profit to mitigate risk in the sites with lower rainfall and higher variability, such as the low-yielding Loxton, Buloke, Cunderdin and Le Hunte-Elliston. Our results confirm an increasing effect of risk aversion from the high- to the low-yielding sites (Fig. 4). Consequently, the maximum cost of risk aversion varied between 72 AUD/ha and 277 AUD/ha in high-yielding Quirindi and between 355 AUD/ha and 739 AUD/ha in low-yielding Loxton, with the results for all other sites falling in between (Table 2, Fig. 4).

3.5 Implications for farmer decision-making

Comparing across the full set of yield, profit, risk and utility metrics between the site practice and the alternative treatments; for example, at the Northam site, it showed an increase of 618 AUD/ha in mean net return, a decrease of 0.17 in CV, no change in the probability of break even, a decrease in downside risk (584 AUD/ha), an increase of 44% in yield and an increase of up to 700 AUD/ha in CE due to risk aversion as a result of adopting the best option in terms of profit-risk-utility, the OptTOS+Var (in this case, however, OptTOS+Var remained the best option regardless of the level of risk aversion, which did not happen in several other sites). In other words, there is a potentially large yield and profit gain accompanied by a fall in risk premium for the Northam farmers when moving from the current average site practice with low inputs (45 kg N/ha and average 1.7 herbicide sprays per fallow) to a more intensive strategy (~ 150 kg N/ha and average 2.5 sprays per fallow). While the magnitude of change varied for the other sites, there was always an advantage in changing the current site practice. Crucially, measuring of the yield gap that can be economically closed by addressing socio-economic constraints interlinked with their agronomic context will boost grower and adviser confidence to predict additional inputs required by crops and the associated economic risks.

The detrimental role of risk aversion in farmer decision making was clearer, however, when focusing on a small subset of agronomic practices where individual elements of the Yw best management practice were set at suboptimal levels (e.g. N45Split2, Fallow Mgt6, SowDelay and Plants75), as reported in Hochman et al. (2019). In this example, only for high-yield potential Quirindi was profit maximising Yw regardless of risk aversion (results not shown). When broadening the range to include the set of emerging practices that have the potential to lift the yield frontier (Sow26April, OptTOS+Var, Ymax), the impact of risk aversion could only explain yield gaps in low-yield potential sites. In medium- to high-yielding areas, we were able to demonstrate that applying the management inputs required to achieve water-limited yield greatly reduced the cost of risk aversion overall, even though ~ 70% of yield (SEYg) was still lost due to other economic factors such as restricting inputs to maximise profitability and farm budget constraints. This highlights the important fact that the capacity of Australian farmers to close the yield gaps while considering profit and risk trade-offs differs according to their water-limited yields.

Overall, there is an opportunity to increase profit and reduce risk at all four levels of farmer risk aversion with higher N applications in combination with better fallow weed management and optimising site- and variety-specific sowing times. Our study builds on the work conducted by others (Cassman et al. 2003; Hochman and Horan 2018; Van Dijk et al. 2017; Van Ittersum et al. 2013; Zhang et al. 2019) on the importance, magnitude and cause of crop yield gaps by overlaying an economic risk aversion analysis to their conclusions. We confirm the role that risk and risk aversion play in many cropping regions by reducing investment in inputs that consequently reduce average production and profit (Monjardino et al. 2013, 2015). Recognizing this fact adds value to farming systems research by explaining the rational decisions of farmers with different levels of risk aversion seeking to maximise their utility and by helping to identify practice change with specific advantage to risk-averse farmers. This work helps to consolidate the valuable role of integrated modelling in yield gap analysis through better understanding of the relative contributions of genotype, environment, management and socio-economics—the G × E × M × S paradigm underlying global crop research (Reynolds et al. 2018).

With yield gains of more than 0.5 t/ha in moving from low-input current practice to profit-maximising strategies, across Australia’s 14 million hectares of wheat fields, let’s assume that on 7 million hectares 0.5 t/ha increase in yield can be achieved. At the farm gate, this would be an increase in grower revenue of 735 million AUD p.a. This translates into some 300 million AUD p.a. in additional profit (after paying the extra input costs) or 43 AUD/ha additional profit.

Surprisingly, even under high levels of risk aversion, for all 14 sites, there are practices with higher risk-adjusted profit than the site practice treatment. This means that, while it is accepted that attitudes to risk are often ingrained and difficult to change, farmers could nevertheless attempt to seize the missed opportunity of achieving significantly higher yield through better understanding of its drivers and more accurate quantification of its costs. Alternatively, farmers may be able to make more informed decisions on the adoption of risk-mitigating solutions such as Multi-Peril Crop Insurance (MPCI), or a novel supply chain contract (Anderson and Monjardino 2019).

Further, the risk-adjusted profit of the emergent best practice of optimising time of sowing and variety maturity as demonstrated by Flohr et al. (2017) exceeded both site practice and Yw at all 14 sites, and this indicates that all farmers could safely strive to adopt this practice even though it is not always the optimal risk-adjusted practice. In some medium and higher yielding sites, the full Emergent Best Practice package (including additional N fertiliser to ensure N-unlimited yield) has the most risk-adjusted profit for some levels of risk aversion and this suggests that at least some farmers in these regions may strive to achieve this yield level.

4 Conclusion

This analysis investigates how profit-risk trade-offs and farmer risk aversion are likely to impact on a range of agronomic practices to close the socio-economic yield gap across five farm risk profiles in 14 sites at seven subregions across Australia. We were able to demonstrate for the first time the profitability of applying a mix of non-limiting N fertilisation, in-fallow weed management and/or optimised time of sowing in closing yield and profit gaps, while mitigating risk across a range of farmer attitudes to risk. Adopting an emergent practice of optimising the site-specific time of sowing and matching variety (maturity type) to time of sowing led to improved yields and profit-risk profiles relative to current best practice for most levels of risk aversion. A key influence on these results is the farmer being able to select from a range of available varieties based on seeding time/maturity times.

Overall, there is a large difference in the cost of risk aversion between low rainfall (low producing) and high rainfall (higher producing, more reliable) sites and only at the low-rainfall site does the ‘best’ practice become affected by risk aversion. This study decomposed economic and agronomic yield gaps and demonstrated how their relative importance interacts with their agro-ecological context. This knowledge is relevant from an industry perspective to inform agricultural recommendations and from a policy perspective because knowing the sources of the yield gap can be used to inform targeted policies.

Importantly, our results emphasise the need for employing a range of research tools, such as crop growth simulation models in combination with profit-risk measures and risk aversion theory to help identify and reduce yield gaps and uncertainty in crop management across different agro-ecological zones, both in Australia and around the world.