One simulation, different conclusions—the baseline period makes the difference!

In the context of climate change mitigation and adaptation, decision-makers generally call for information about impacts of projected changes in a specific region at different global warming levels or in certain future periods. They need answers to questions like: 'Can we expect an increase or decrease in water availability, extreme events, such as floods, droughts, storm surges or heatwaves, around the year 2030, 2050 or by the end of the 21$^{\rm st}$ century? And what will be the consequences for, e.g. crop production, renewable electricity generation?' To answer such questions, regional climate impact modelers face a variety of challenges, which relate to technical, methodological, and communication issues of simulation results [1] and corresponding recommendations under uncertainties in a comprehensible way.

Adding to technical and methodological challenges includes, e.g. the choice of climate scenarios, climate and impact models, the use of bias-adjustment methods, and model calibration and validation periods. The performance of a climate model is usually measured against its ability to represent spatial patterns and trends in the historical climate. Sometimes the performance is used to assign weights to individual models within a model ensemble [2–7]. The uncertainty cascade in the impact modeling is basically associated with model structure, model parameterization, and input data quality [8–15].

After the simulations have been carried out, the question about the baseline period used to compare future simulation results to, will arise. Where future scenario periods are usually defined to reflect the decision maker's planning horizon, baseline periods are often chosen arbitrarily or are based on existing standards. However, choosing a baseline period is a sensitive issue and can be easily instrumentalized to support specific conclusions, whether intentionally or not.

The World Meteorological Organization (WMO) recommends to use the 30-year period of 1961–1990 as the climate normal when comparing with future periods and that this should be maintained as a reference for monitoring long-term climate variability and change [16, 17]. Beyond that, a regularly updated 30-year baseline period, currently 1981–2010, should be employed to give people a more recent context for understanding weather and climate extremes and forecasts [17–19]. The Intergovernmental Panel on Climate Change (IPCC) used the 20-year period 1986–2005 as the baseline in many graphs in the Fifth Assessment Report [20] and will use the years 1995–2014 in its Sixth Assessment Report. So, what are climate impact modelers supposed to do? Which baseline should they select and does it actually matter?

At the global or continental scale, it is virtually impossible to choose a baseline period whose climate is represented realistically by all climate models. An arbitrary determination of global baselines is therefore justifiable. However, global and regional climate simulations are often not designed to synchronize with real year to year patterns and events [21], which creates a communication challenge, particularly in regional impact studies. For example, some climate models depict the mid-1980s as a period with above-normal rainfall, when in reality a drought hit West Africa. Others simulate the extraordinary wet 1950s and 1960s as very dry. Nevertheless, well-established global baseline periods are often used unquestioningly in regional impact studies, although the real-life statistical properties of the specific historical period may not be adequately represented by climate model simulations, therefore, also not in subsequent applications.

Even though the implications of the choice of baseline periods for the interpretation of simulation results are well known, little attention has been paid to them in the climate impact community. Ruokolainen and Räisänen (2007) [22] analyze the sensitivity of forecasts to the choice of different baselines in Southern Finland. Razavi et al (2015) [23] emphasize that different length of baseline periods may lead to different conclusions about stationarity/non-stationarity. Hawkins and Sutton (2016) [18] discuss the choice of climate reference periods when comparing global air temperature projected by climate models with observations. Huang et al (2018) [24] depict future flood characteristics in future periods in four river basins based on different 30-year baseline periods. Snell et al (2018) [25] highlight the sensitivity to the choice of baseline climate in dynamic forest modeling in the Alps. Baker et al (2016) [26] assessed the impact of six different climate baselines on projections of African bird species' responses to future climate change. Although this issue has been addressed as a side effect in several other studies, it has generally not been considered important to form the focal point for systematic research.

The present study systematically investigates the effect of the choice of the baseline period on the interpretation of simulation results. It provides examples from eight river basins located in various climate zones, where changes in projected future discharge are estimated based on WMO and IPCC baselines using four bias-adjusted and downscaled Global Climate Models (GCMs) from the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP) [27–29]. An index measuring the similarity between two time series is introduced and was used to assess the sensitivity of choosing different baseline periods. We developed an algorithm to overcome the problem in cases of substantial deviations. It identifies a baseline period, which consists of similar basic statistical properties as the historical period and is flexible in terms of length and timing. Although the main focus of this study is the analysis of river discharge, the method is in principle applicable to any time series variable, such as meteorological data, crop yields, emissions of greenhouse gases, hydropower potentials and so on.

2.1. Study sites

The impact of different baseline periods was investigated by using simulated river discharge from eight exemplary river basins located in various climate zones from equatorial to polar (figure 1 and table 1). The simulations were carried out within the framework of various research projects (see references in table 1). What they have in common is the hydrological model, the same four forcing GCMs, and the simulation period they cover (1960–2099), which guarantees consistency across the study basins.

Zoom In Zoom Out Reset image size

Table 1. River basins.

River basin	Gauge	Area [km2]	Region	Climate zone
Northern Dvina (DVI) [31]	Ust-Pinega	348.000	Europe, Russian Federation	Snow (Dfc)
Rhine (RHI) [24]	Lobith	160.000	Europe	Warm temperate (Cfb)
São Francisco (SFC) [32]	Outlet	640.000	S America	Equatorial (Aw, As), arid (BSh)
Tagus (TAG) [31]	Almourol	70.000	S Europe	Warm temperate (Csa, Csb)
Upper Blue Nile (UBN) [4, 33, 34]	El Diem	175.000	E Africa	Arid (Aw), warm temperate (Cwb)
Upper Mississippi (UMI) [24]	Alton	440.000	N America	Snow (Dfa, Dfb)
Volta (VOL)	Outlet	403.000	W Africa	Equatorial (Aw), arid (BSh)
Upper Yellow (YEL) [24]	Tangnaihai	121.000	China	Polar (ET), snow (Dwc)

2.2. Data

2.3. Baseline periods

2.4. Similarity index

The MQ time series was used to compute the relative change between a specific baseline and a corresponding future period as follows:

$$\begin{equation} \Delta MQ_{i,j} = \frac{\overline{MQ}_{future,i} - \overline{MQ}_{base,j}} {\overline{MQ}_{base,j}}, \end{equation} \tag{ 1 }$$

where $\overline{MQ}_{base}$ is the average of the annual values of a specific baseline period and $\overline{MQ}_{future}$ the average of a future period. The index i refers to different future periods with central years between 2020 and 2080, i.e. 61 time steps. The index j represents the different baselines (WMO, IPCC, and flexible baseline), where the length of the baseline determines the number of years around the central years in corresponding $\overline{MQ}_{future}$ periods.

The mean absolute deviation between two ΔMQ time series, e.g. ΔMQ_WMO,i for the WMO and ΔMQ_IPCC,i for the IPCC baseline, over all k = 61 time steps was then quantified as:

$$\begin{equation} D = \frac{1} {k} \displaystyle\sum_{i = 1}^{k} \mid \Delta MQ_{WMO,i} - \Delta MQ_{IPCC,i} \mid. \end{equation} \tag{ 2 }$$

The deviation was then re-scaled by a user-defined deviation threshold $D_{\textrm{max}}$ to an agreement score value

$$\begin{equation} AS = \begin{cases}1 - \frac{D}{D_{\textrm{max}}} & {\rm if} \quad D\leq D_{\textrm{max}}\\ 0 &{\rm if} \quad D > D_{\textrm{max}} \end{cases} \end{equation} \tag{ 3 }$$

This Agreement Score ranges between one (no deviation, perfect agreement) and zero (deviation larger than the threshold). In this study a threshold value of $D_{\textrm{max}} = 25 \%$ was defined, because deviations in discharge projections > 25% that are solely based on different baselines, were considered to be very large and indicative for a substantial difference. For other applications (e.g. greenhouse gas emissions, temperature, precipitation, wind speed) or by using not relative but absolute changes for ΔMQ_i,j, other threshold values might be more appropriate.

Apart from the deviation based on the choice of different baselines, we quantified the direction of change signals CS as

$$\begin{equation} CS_{i,j} = \begin{cases} 1 & {\rm if}\ \Delta MQ_{i,j} > 0.01\\ 0 & {\rm if} -0.01 \leq \Delta MQ_{i,j} \leq 0.01,\\ -1 & {\rm if}\ \Delta MQ_{i,j} \lt -0.01\end{cases} \end{equation} \tag{ 4 }$$

compared the agreement between two baselines for the future periods by setting

$$\begin{equation} AC_i = \begin{cases} 1 & {\rm if}\ CS_{WMO,i} = CS_{IPCC,i}\\ 0 & {\rm if}\ CS_{WMO,i} \neq CS_{IPCC,i}\end{cases} \end{equation} \tag{ 5 }$$

to eventually derive the average agreement in the direction of the change signal by

$$\begin{equation} AC = \frac{1} {k} \displaystyle\sum_{i = 1}^{k} AC_i. \end{equation} \tag{ 6 }$$

Finally, a Similarity Index was defined as

$$\begin{equation} SI = \frac{AS + AC}{2}. \end{equation} \tag{ 7 }$$

A value of zero is derived if the selection of a baseline has a large impact on the interpretation of results of an impact study while the optimum value of SI = 1 is achieved if the choice of the baseline would have no influence at all. An intermediate value of SI = 0.5 can be derived if the absolute deviations are large with respect to the user-defined $D_{\textrm{max}}$ value, but both baselines show the same directions of change over all possible future periods. Likewise, a value of approximately 0.5 is obtained when the choice of different baselines results in a bias with small absolute deviation with respect to $D_{\textrm{max}}$, although the directions of change deviate for all possible future periods.

The computation of SI was also tested by integrating other factors, such as agreement in standard deviation or R², but the results achieved with a more complex indicator were not considered to be more meaningful than those achieved with the simplistic approach. The SI was also used to assess the sensitivity of the choice of the baseline depending on the GCM and the climate zone.

This section shows to what extent the choice of the baseline alone can influence the interpretation of simulation results.

Figure 2 shows future ΔMQ series for selected river basins relative to MQs in the WMO and IPCC baselines. Future change signals and magnitudes of change can be extremely different between the two ΔMQ series (figures 2(a) and (c)). Both examples are therefore characterized by low SI values of 0.24 and 0.19, respectively, which indicate large differences of MQ values in the respective baselines. They also demonstrate that neither the results based on the one nor the other baseline generally tends to suggest higher or lower future ΔMQ, a phenomenon also found in river basins shown in appendix C. Considering the example of the Northern Dvina River basin (figure 2(c)), one would conclude that future ΔMQ does not change substantially but rather fluctuates around the historical mean if the IPCC baseline is used. A completely different conclusion would be drawn with the WMO baseline, where ΔMQ is projected to increase between 22 and 32%. This illustrates how the choice of the baseline period, based on the same model simulation, would lead to conflicting recommendations for adaptation strategies.

Figures 2(b) and (d) show examples of future ΔMQ where it apparently does not matter which baseline is used as reference. The corresponding SI values of 0.97 and 0.84 are therefore much higher than in the other two examples. Recommendations for adaptation strategies would consequently be much less dependent on the choice of the baseline period in these cases.

Zoom In Zoom Out Reset image size

A visual assessment of the ΔMQ series with the corresponding SI values in figure 2 is conclusive, where low SI values indicate a high sensitivity to the choice of the baseline period and high SI values a low sensitivity. As with model performance indicators (e.g. R², PBIAS), an evaluation of which value ranges indicate actually a good or poor fit, or in the case of the SI, which values represent high or low sensitivity, remains somehow subjective. In the context of simulated river discharge, we propose SI values below 0.5 to indicate high sensitivity.

The choice of the baseline period has the highest impact on the interpretation of simulation results performed with the IPSL model and the lowest impact with the MIROC5 model. However, the average GCM SI value (table 2) does not imply that this assumption is true for all basins and all RCPs. The results for RCP 8.5 are slightly different, where the highest SI value is also achieved with the MIROC5 model, but the lowest values with GFDL (table D1).

Assuming an SI threshold of 0.5, it mattered in 25% of the simulations under RCP 2.6 (table 2) and in 32% under RCP 8.5 (table D1), whether the one or the other baseline was used to assess future changes. There are basically two options to deal with simulations resulting in SI ≤ 0.5: (i) discuss the uncertainties and/or (ii) choose a different baseline that represents the basic statistical properties of the historical period more consistently, e.g. by using the proposed algorithm in appendix A.

To exemplify, the projected discharge changes with an additional flexible baseline for the Volta River basin is shown in figure 3. It explains why the results derived with both baselines are so different. The MQ values for the WMO and IPCC baselines are 1454 and 1115 ${\rm m}^{3}{\rm s}^{-1}$, respectively. The algorithm identifies the 30-year flexible baseline 1972–2001 with an MQ value of 1361 ${\rm m}^{3}{\rm s}^{-1}$, which is much closer to the MQ of the WMO than to the MQ of the IPCC baseline. Furthermore, the range of values of the IPCC baseline is much smaller. Where the years with the lowest discharges are identical, the highest MQ is only 1800 but 3150 ${\rm m}^{3}{\rm s}^{-1}$ in the WMO and flexible baselines. This would make a significant difference in an analysis of the distribution of wet, dry, and extreme years.

Zoom In Zoom Out Reset image size

Results from all river basins under both RCPs show that the projected ΔMQ series using flexible baselines lie either in between or outside WMO and IPCC ΔMQ. But, in all cases, they resemble the WMO more than the IPCC ΔMQ series (figures in appendix C), which is an indication that also the length of the baseline period matters.

Table 3 shows relative differences of ensemble ΔMQ between WMO, IPCC, and flexible baselines for all river basins around the central years 2040, 2060, and 2080 for RCP 2.6 and table E2 for RCP 8.5. In the Northern Dvina River basin (DVI) in 2040 and 2060 and in the São Francisco River basin (SFC) in 2080, the ensemble mean projects opposing change signals between WMO and IPCC baselines, with absolute differences up to 13.9% under RCP 2.6 and almost 20% under RCP 8.5. Relative differences between WMO and IPCC baselines are lower if the ensemble mean is considered (appendix E), but can be very high for individual models, as was shown in figures 2(a) and (c). As with individual models, the ensemble mean ΔMQ series of the flexible baseline are always more similar to the WMO than to the IPCC ΔMQ series.

The sensitivity (SI ≤ 0.5) of the choice of the baseline period for different climate zones is inconclusive (table 2 and table E2). A larger sample size of catchments from various climate zones is required to make more robust statements. However, the lowest sensitivity was achieved in warm temperate climates (C) represented by the Rhine, Tagus, and Upper Blue Nile River basins.

This study demonstrates how solely the choice of a baseline period can influence the interpretation of discharge projections in eight river basins using climate input from four bias-adjusted GCMs. To evaluate whether the choice of either the well established 30 years (1961–1990) WMO [17] or the more recent 20 years (1986–2005) IPCC [20] baseline matters, a similarity index SI was introduced as a measure to compare the two resulting time series of future change. In about 25% of the simulations under RCP 2.6 and in 32% under RCP 8.5, large quantitative differences and/or opposite signals of change were found, with at least one case of major discrepancies in each river basin. The deviations for selected future periods can be so large that they range from −5% to +45% for a given central year. These figures indicate that different recommendations for action could possibly be derived in at least every fourth case.

No systematic differences in the direction of change using either baseline period could be identified. Neither the results based on the WMO nor those based on the IPCC baseline tend to generally project higher or lower future river discharge.

4.1. Choosing baseline periods

4.2. Regional context

4.3. Ensemble mean versus single model simulations

4.4. Outlook

Acknowledgment

C.1. RCP 2.6

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size