Introduction

Scenarios can be a useful tool to support policy and guide action towards sustainability [1]. In order to do so, potential sustainability impacts of different scenarios need to be assessed. A number of methods can be used for sustainability assessments (SAs), depending on the purpose, focus and scope [2,3,4,5]. Important features of many methodologies under the umbrella of SAs are, among others, the importance of considering both environmental and social aspects, of acknowledging values brought into the assessment, consideration of how sustainability is viewed, uncertainty and transparency handled and the importance of participatory approaches [6,7,8,9].

Based on Börjeson et al.’s typology [10], we consider three types of scenarios: predictive, (forecasts and what-if scenarios), explorative (external and strategic scenarios), and normative scenarios (preserving or transformative (e.g. backcasting)) answering the respective questions:

  • What will happen?

  • What can happen?

  • How can a specific target be reached?

The role of scenarios in different sustainability assessment methods has been discussed (e.g. [11]). The role of sustainability assessments for evaluating scenarios has however been less discussed, although the importance has been acknowledged. For example, Robinson [12] has highlighted the need to include impact assessments in the backcasting process as the last of a six steps’ iterative process; a step which, according to the author, has been the most neglected. Even if some aspects may be assessed as a result of the scenario analysis itself, Robinson continues, not all relevant sustainability aspects are considered and second-order effects are usually left out.

The aim of this paper is to explore methods for assessing the sustainability impacts of future scenarios, trying to answer the question ‘What can we learn for the further development of sustainability assessments of scenarios?’ A starting point of the paper is a literature review of case studies, looking for methods that have actually been used for assessing sustainability impacts of scenarios, in addition to papers describing possible methods. The focus is on assessment of environmental and social impacts, as it has been argued that sustainability can be defined as a safe and just operating space for humanity with an environmental ceiling and social foundation, while the economy may be seen as a means to stay within that space [13]. However, some aspects lie at the frontier between the social and economic domains and are to some extent considered. One type of scenarios are plans, which here encompass binding and non-binding municipal strategy documents [14], with a spatial representation of potential development policies that can extend over a few years or decades. Here, assessments of plans have been analysed separately as they are viewed as a field of its own. Scenarios can be particularly relevant, yet challenging, for sectors that develop fast. A special focus is put on sustainability assessments of scenarios in the Information and communication technology (ICT) sector, seen as an example of a rapidly developing sector.

The number of sustainability assessment tools that can be used for analysing scenarios is potentially very large. We have not aimed to identify and describe all. Instead, the focus is on tools that are used in practice and can be useful in different applications.

A note on terminology: There is no common agreement in literature about any difference in the use of terms such as ‘tool’, ‘method’ and similar with regards to sustainability assessments. Sometimes these terms are used interchangeably [15]. In this paper we use “method” and “tool” as synonyms.

Method

In order to identify relevant methods, a literature search was performed using keywords such as “social impact”, “assessment”, “sustainability”, “scenarios”, and “future” in different combinations. The choice was delimited to studies from 2005 and later. Additionally, studies known from previous experience were screened and included if relevant.

Specifically developed for the assessment of plans, (and also required by law in many countries for some types of plans) [16], Strategic environmental assessment (SEA) is a procedural method which can include several types of analytical tools [17, 18]. Tools described as suitable in SEA guidelines and/or used in practice for SEA are also considered in the overview of methods. For SEA, several guidelines [17, 19,20,21,22] were reviewed for suggestions on analytical tools that can be used for assessing plans. In addition, several recent plans in the Stockholm region were reviewed in order to analyse which methods are used in practice. Representatives from Stockholm City, Stockholm County and Täby municipality also participated in a workshop for more in-depth discussions. In the end, eight SEAs for Swedish municipal plans were analysed [23,24,25,26,27,28,29,30] (see Table 2 in the appendix).

The following questions were considered when analysing the assessments of both scenarios and plans:

  • Which sustainability aspects are addressed?

  • What type of assessment is used (qualitative/quantitative)?

  • Which tools are used to assess sustainability aspects?

  • If ICT is addressed in the assessments, how is it addressed?

  • What is the time frame for the scenario or plan?

Description and analysis of tools suggested and used

Tools can be divided into procedural tools focusing on the procedure and the decision context and analytical tools focusing on the analysis of impacts [2]. We also consider tools to aggregate impacts and tools that contribute to both the scenario building and assessment. An overview of the reviewed papers with social and environmental impacts considered can be found in Table 1.

Table 1 Overview of tools and aspects addressed in the reviewed literature
Full size table

Procedural tools

SAFS – Sustainability assessment framework for scenarios

Sustainability assessment framework for scenarios (SAFS) is a methodological framework describing the procedure for the qualitative assessment of future scenarios [31]. The framework is designed for assessments on a societal level with a consumption perspective and life cycle thinking, addressing a range of environmental and social aspects. The consumption perspective implies that the starting point is consumption within a society, in contrast to a production perspective. Life cycle thinking implies that impacts occurring during the whole life cycle should be considered including the production of products and services, the use phase and waste management, wherever they occur.

SAFS consists of the following steps: scoping, inventory analysis, assessment of risks and opportunities, and interpretation. In the scoping, the goal and scope of the study are defined and the aspects to be addressed are decided upon. The inventory analysis includes gathering relevant information from the scenarios as well as collecting data on current state of the chosen aspects. Assessment of risks and opportunities consists of three sub-steps. First, an analysis of interrelation between various factors described in the scenarios and environmental as well as social impacts is done based on factual knowledge, literature and workshops with experts and stakeholders. Then an assessment of risks and opportunities is carried out through discussions in an assessment team based on the outcomes of the first step. Last, the results for environmental and social aspects are integrated. Finally, the results are interpreted.

SAFS was developed to be applicable in various contexts and timeframes; however, its qualitative approach makes it best fitted for assessing long-term scenarios with large transformative changes [31]. For short-term assessments, a quantitative approach may be preferred, as it allows for getting a more precise and easily communicated result. On the contrary, qualitative tools such as SAFS can accommodate for the high degree of inherent uncertainty characterising long-term scenarios.

SAFS doesn’t provide the list of aspects to address but provides guidelines for defining those aspects and emphasizes the importance of addressing both environmental and social aspects. It was applied for assessing scenarios of future Swedish ICT societies in 2060 [31], against both environmental and social aspects.

Sustainability impact assessment (SIA)

Sustainability Impact Assessment (SIA) (not to be confused with Social Impact Assessment, also abbreviated SIA) is described in Martire et al. [32] as a process-oriented tool and aims at assessing the sustainability effects based on environmental, social and economic indicators of changes on sector level, using baseline and scenario descriptions. It allows for tailoring specific system boundaries, and is flexible in how to merge various data as well as how to model local supply chains. The SIA approach - and the linked tool Tool for Sustainability Impact Assessment (ToSIA) - was applied on the development of local energy supplies in a region in Italy within a rather short-term (2020) [32]. ToSIA is a tool for comparing scenarios with the baseline and/or against each other, based on quantified impacts and was originally developed for the forestry sector. In this application, only a limited number of indicators were considered. The indicators chosen were selected from the ToSIA framework and decided upon based on local conditions. Indicator data was calculated manually, based on the instructions in the tool and assessed quantitatively.

Framework for participatory impact assessment (FoPIA)

Several of the assessments reviewed included participatory approaches. One tool specifically looking at how to ensure participation of stakeholders is the Framework for Participatory Impact Assessment (FoPIA), which provides a template to facilitate the involvement of national, regional and local stakeholders in assessing sustainability impacts. König et al. [33], used FoPIA to assess land use scenarios in developing countries. It consists of a preparatory phase, a stakeholder workshop and an evaluation phase. The scenarios were explorative land use scenarios with different timeframes from 2015 to 2030. The workshop was structured to follow certain steps, whereof ‘specification of the sustainability context’ and ‘assessment of scenarios impacts and analysis of possible trade-offs’ are two. The assessment was made qualitatively by assigning score in the scale −3 to +3, thus considering both negative and positive impacts.

Analytical tools

Qualitative mapping of sustainability impacts and goal conflict analysis

One way of mapping sustainability impacts is by using a checklist. This is a tool that helps identify and avoid overlooking relevant issues when analysing the consequences of a certain measure or decision [69]. If different scenarios are assessed at the same time, this may be done in a matrix approach, with questions for the checklist on one axis and the scenarios on the other.

Differences between various checklist approaches include; what questions are addressed, who conducts the analysis, the analysing method, whether a comparison is made between scenarios and the way the results are presented (e.g. in written text or symbols for example smileys, numeric scores). One common way for presenting results is the “value-rose chart”, sometimes referred to as “spider chart”. The chart consists of a circle and different radiuses representing various sustainability aspects. This tool helps visualise the different performances on various sustainability indicators in the scenarios.

Baard et al. [34] suggest the use of a checklist in municipalities for their climate adaptation plans as a first step. This was applied in the assessment process of identified climate adaptation measures and their implications in different explorative socio-economic scenarios combined with climate scenarios. Two planning horizons were considered: 2030 and 2060. The checklist offers nearly 30 questions to choose from, covering all three dimensions of sustainability. The list includes specific questions about distribution and ethics as a way to identify the potential winners and losers (individuals or groups) of a measure over time. As a second step, a mapping of the consequences on the three sustainability dimensions was carried out in a participatory workshop [34], where planners from different sectors listed the consequences of different measures in the scenarios, and estimated the magnitude of the consequences qualitatively according to the scale low/medium/high decrease or increase.

Similar semi-quantitative assessments are sometimes used in SEAs. One example is the SEA for the comprehensive plan for Täby Municipality [29] where expected impacts were compared to the current situation and illustrated in a value rose chart. The assessment used mostly quantifiable indicators, while some were experience-based. It compared different alternatives in terms of their environmental sustainability to the current situation according to a scale (from −4 to +4). The time horizon was 2030.

Kowalski et al. [35] developed five explorative scenarios focusing on sustainable energy futures for Austria in 2020, which were assessed using a life-cycle approach against 17 sustainability criteria. In addition to quantitative environmental indicators mentioned in the section below on LCA, social and economic aspects were assessed qualitatively ranging from low to high through expert interviews.

Goal conflicts and synergies can be identified by linking the aspects included in a checklist to different types of policy goals. This can be useful in order to assess whether a certain measure aimed at reaching a specific target may at the same time facilitate or hamper the achievement of other targets. Such approaches were used in several academic examples [34, 36, 42] but also in several municipalities’ plans [27, 28, 30] and in a report from the Swedish Energy Agency [37].

In a report looking at scenarios for the Swedish energy system in the mid-term (2035) and longer term (2050), four explorative scenarios were developed with varied given importance to climate, environmental and resource preservation or energy security for the industry [37]. For each scenario, an assessment was made against the national environmental goals [70, 71]. The impact of different scenarios was expressed in risk for an increasing negative impact or opportunity for an increasing positive impact on a goal. The results were presented in text form. In the same study, the scenarios were also roughly assessed against the EU 2030 climate goals, various national goals addressing climate issues and the Paris Climate agreement [72], considering how far or close the different scenarios were to these goals. Finally the 17 Sustainable Development Goals (SDGs) [73] were mentioned and for each scenario, one SDG was singled out as being particularly facilitated by the specific drivers behind each scenario.

In Svenfelt et al. [36] a goal conflict analysis was carried out in order to assess the impact of different measures illustrated in backcasting scenarios for land use in Sweden that all aimed at reaching a climate target of zero net Greenhouse gas (GHG) emissions in 2060. This study explored potential conflicts with other environmental goals on national level [70, 71]. The analysis was summarised in a compatibility matrix using colours and showing potential synergies in green, potential conflicts in red and too great uncertainties in white [36].

In Sheate et al. [38], a qualitative assessment of the agricultural restructuring and decline for biodiversity conservation in mountain areas in Europe was done by scenario analysis, supported by the views from stakeholder panels all along the work. Six European regions were studied using explorative scenarios for 2030. Flow diagrams were developed for disaggregated individual drivers, showing the interaction between drivers and impacts. Sustainability objectives were drawn from externally set objectives on national and international/EU-level. The assessment was done qualitatively, following the defined flow diagrams for the interaction between drivers and impact, assessing the outcome for the impact using a five point scale from “++” to “--“.

Cartmell et al. [39] developed scenarios of different approaches for bio solid co-combustion in the UK. The level of risk for four broad indicators was determined by interviews with relevant stakeholders. The relative performance of the options on these indicators was then assessed qualitatively using a simple scoring system ranging from −2 (very negative), through 0 (neutral, or balance of negative and positive), to +2 (very positive). Each score on the scale was defined for the four different indicators by a short descriptive text.

Network analysis

Network Analysis (NA) is a tool that analyses the relationships between the entities within the area of the analysis, rather than the entities themselves. That may be for example how different actors interact with each other, or how different actions relate and influence each other. NA is seen as helpful to assess real world systems in which drivers do not act in isolation, where there might be several different impacts and consequences, and to understand which entities are key in the system [41]. In Tzanopoulos et al. [40], NA was used on explorative scenarios in a 25 years perspective, assuming different future agricultural policies on a national level (Greece). The casual relationships between drivers of change, their impacts and performances in relation to sustainability objectives are analysed and NA allows for the identification of critical paths, and entities that control the flows. The steps in the assessment consisted of a baseline assessment of past and current situation, and an identification of sustainability objectives. After generating the scenarios, the sustainability assessment was done at a cross-disciplinary stakeholder meeting; the scenarios were dis-aggregated and the causal relationship between the drivers of change and the impacts and consequences were assessed qualitatively, against the agreed set of objectives.

Also in Boron et al. [41], NA is used. The scenarios are assessed against environmental objectives. The approach consisted of interviewing stakeholders in order to understand the drivers of change and their impacts on sustainability in the local context, to review scenarios and to identify sustainability objectives for the study. The selected sustainability objectives were also based on policy documents.

Life cycle assessment

Life cycle assessment (LCA) is a tool to assess the potential environmental impacts and resources used throughout a product’s or service’s life cycle from raw material acquisition through manufacturing, use to final waste disposal [74]. The aim of an LCA is to assess the potential impacts from a system’s perspective to avoid sub-optimization and problem shifting when identifying strategies for improvement [75]. LCA consists of four main steps – goal and scope definition, inventory analysis, impact assessment and interpretation [74]. LCA is a quantitative method requiring collection of detailed data although life-cycle thinking can be used as a qualitative approach.

Two types of LCA are distinguished – attributional and consequential (e.g. [76]). Attributional LCA addresses the question “What are the environmental impacts of product/service A?” and uses average data to assess the product system [77]. Consequential LCA addresses the question “What are the environmental consequences of producing and using x extra units of product/service A?” and uses marginal data to assess the potential environmental impact of the change in the system [77].

LCA is traditionally applied to a product or service, however, some studies use LCA or life cycle approach for scenario assessment. De Camillis et al. [78] looked at how LCA could be used for assessing prospective, explorative and backcasting scenarios from near- to long term. The authors suggest that different types of LCA could be used depending on the question in focus. They also argue that Macro life cycle assessment (M-LCA), combining the general equilibrium economic modelling, further described in the section below on Computable General Equilibrium models, and LCA, is particularly suitable to compare different future-oriented scenarios.

LCA was used for scenario assessment in a number of studies reviewed [35, 42,43,44,45,46,47,48,49,50,51, 65, 78]. All of these studies consider mainly environmental impacts and look at either a sector, technology or policy on a national, regional or global scale with different time perspectives (from near to long term).

Nilsson et al. [42] used attributional LCA for evaluating different alternatives within an SEA of a waste incineration tax proposal. Björklund [50] used LCA in SEA of municipal energy planning. Qualitative data on future scenarios had to be expressed quantitatively when performing the LCA. The study applied both types of LCA: attributional and consequential. Attributional LCA was used to compare the current state to the no-action alternative and consequential LCA was used to compare each proposed measure to the no-action alternative.

M-LCA is used in e.g. Dandres et al. [45] to assess global environmental implications of policy implementation in the EU energy sector. Münster et al. [49] discuss two cases of LCA application for future scenarios assessment – assessing policies related to waste management using attributional LCA [65], and scenarios for technological development of renewable energies using consequential LCA [49]. Both studies used LCA in combination with economic modelling. Several studies used LCA to explore environmental implications of implementation of technologies (CCS and transport electrification) and alternative energy generation [43, 47, 51]. Malmodin and Bergmark [48] used an LCA to explore the effects of ICT on GHG emissions in future scenarios. Chen et al. [44] point out that consequential LCA is good for strategic analysis of choice of technologies to be used in the future, being aware of uncertainties induced by various factors. The authors suggest a graphical representation of the analysis aiming at simplifying the communication for decision-makers. Santoyo-Castelazo and Azapagic [46] suggest using LCA within their decision-support framework for assessing energy systems. Berrill et al. [52] used LCA for analysing energy scenarios for Europe in 2050.

Social life cycle assessment

Social Life Cycle Assessment (S-LCA) is a rather novel method, aimed at assessing social impacts from products and services all along the product life cycle [79]. It is based on LCA, using the same methodological approach, but addressing social instead of environmental impacts. S-LCA has been used for assessing scenarios in a few case studies. Scenarios for different combinations of treatment methods for used PET bottles were assessed [53]. These scenarios were of a short-term character, rather assessing different combinations of treatments than long-term developments. The assessment was done in alignment with the Guidelines for S-LCA [80]. Data collection for the selected social aspects, chosen from the aspects listed in the guidelines, but limited to those deemed relevant for the topic, was largely based on interviews during which questionnaires were filled in by different actors such as scavengers (private and state) and different categories of workers (landfill, incineration and flake production industrial) capturing their views on the social performance for the different industries.

Assessing the energy system in Luxemburg in a prospective approach, Rugani et al. [54] used data from the Social Hotspot Database, the first database for social data on social issues on country and, if available, sector level [81]. The social impacts were assessed using S-LCA in a Business as usual (BAU) future scenario based on projections, up till 2025. They used an input output (I/O) approach (see the section below on Inout-output analysis) and the basis of the assessment was the projected energy demand. A LCA was also conducted on the scenario in this study.

Life cycle sustainability assessment (LCSA)

Life cycle sustainability assessment (LCSA) is the combination of three separate life-cycle based assessments of environmental, social and economic impacts from products or services; LCA, S-LCA and LCC (Life Cycle Costing). LCC is a method for addressing the life cycle based costs for the products or services considered, thus including costs for operation, service, maintenance and waste. In some approaches, only producer-related costs are considered, in other also costs borne by other actors and sometimes the whole society are included [82, 83]. It was applied on scenarios on electricity generation in a long term perspective (2070) [55].

Input-output analysis

Input-output analysis (IOA) is an economic tool describing the whole economy in a society including transactions between sectors [84]. An IOA can be used for identifying environmental impacts by adding emission coefficients to the monetary IOA. Results can be presented for sectors and for broad product groups [2, 85, 86]. IOA can be used as a tool for LCA inventory, when information on resource use and emissions for each sector are included [76].

Multi-regional input-output analysis (MRIO) allows for including trade between different regions as well as geographic differentiation of economic and environmental aspects. It is a rapidly developing field. Its use for calculation of global environmental footprints, and in discussions and analysis of climate policy issues, has made a significant contribution to the field’s development [87,88,89,90]. An example is EXIOBASE [87] which aims to integrate the economy-wide material and energy flow accounting with MRIO analysis and includes multi-regional system of supply-use tables based on statistical data, linked via trade. This database integrates data from environmental accounts, covering resource input (energy, materials, water and land) and waste and emissions outputs. It allows different types of analysis from product LCA to environmental footprints and economy-wide flow accounting [87].

Traditionally I/O modelling is used for accounting studies. Recently there have however been some studies where it is used for scenario modelling (e.g. [91, 92] but also for evaluating social, economic and environmental impacts of future scenarios [56]. The methodology is however still under development and applying it for scenario assessment can be challenging, as I/O analysis requires detailed data, which may be difficult especially for scenarios with transformative changes.

Tools for aggregating impacts

The above-mentioned tools are used to assess environmental or social impacts of scenarios. However the different scenarios or plans may be difficult to compare to each other as some scenarios might prove to be better in some aspects but worse in some others. In some cases it may therefore be useful to further aggregate the results.

The mainly qualitative results from the checklist and matrix approaches described above, are sometimes turned into semi-quantitative approaches by adding the positive and negative impacts, or plus and minus signs. Multi-criteria decision analysis and cost-benefit analysis are families of tools that can be used for aggregation of different types of impacts.

Multi criteria decision analysis (MCDA)

Multi Criteria Decision Analysis (MCDA) (or Multi Criteria Assessment (MCA)) is a family of methods that "evaluate(s) alternative options against several criteria and combines these separate evaluations into one overall evaluation" [34: 102]. MCDA is typically "used to assess and rank alternative options in an impact assessment, or to assess the extent to which a variety of objectives have or not been met, in a retrospective evaluation or fitness check" [93]. The aim of MCA is "the identification of compromise solutions in a transparent and fair way" [36: 1068]. Indeed, MCDA is a tool where the values and the priorities of the decision-makers are made explicit. It can also in its different versions handle data in different formats (e.g. qualitative and quantitative) as well as uncertain data.

This method is used as a second step, once other tools have been used to calculate the impact for an environmental or social aspect (e.g. GHG emission), MCDA can be used to rank scenarios against one another by allocating weights to the different aspects.

In Kowalski et al. [35], a multi criteria analysis was done in a participatory way (Participatory Multi Criteria Assessment - PMCA) using the Promethee (Preference Ranking Organisation Method for enrichment evaluations) method. This method is based on partial aggregation and compares alternatives by pair for each criterion and ranks them [94]. The weighting of the criteria was done by interviewing stakeholders and gathering their individual ranking of the criteria according to their importance (SIMOS method).

Anderson et al. [57] use MCA to assess the sustainability impacts of five backcasting scenarios in which CO2 emissions from the whole UK energy sector are reduced by 60% by 2050 and to discuss the relative performance of the different scenarios leading to the conclusion that a scenario with low energy demand would have less negative impacts on other sustainability criteria, whether environmental, social or economic.

Santoyo-Castelazo and Azapagic [46] apply MCDA to the case of electricity supply in Mexico. They first analyse 11 different explorative scenarios for 2050 with various technologies, electricity mixes and reduction of climate emissions. After assessing the scenarios through a life cycle analysis against 17 sustainability criteria (ten environmental, three economic and four social indicators), no conclusion could be drawn as to which scenarios were most sustainable. The authors therefore conducted a multi-criteria decision analysis against the environmental sustainability criteria and two economic criteria. The social criteria were discussed in connection with the MCDA results. The MCDA was first done by weighting these 12 criteria evenly. Then, as a sensitivity analysis, and to reflect the different preferences of the stakeholders involved (such as industry or government), different weights were given to three most important criteria across stakeholders, i.e., climate change (GWP), human toxicity (HTP) and costs (annualised) [46].

In many cases, MDCA is used for ranking technology scenarios, in which different developments of a specific technology are assessed. For example, in Onat et al. [58], alternative electric vehicle technologies in a US context were assessed including all life cycle stages; from manufacturing of vehicles and batteries, through vehicle operation to the end-of-life. There were two scenarios; one with BAU electricity generation and one assuming only solar charging station for the electric vehicles.

Transport issues were also addressed in López et al. [59], where the use of scenarios aimed at assessing the impacts from transport strategies on energy consumption at a European level. The strategies were of two kinds; technology improvements in vehicle technologies and fuels, and measures to control transport demand. Eight scenarios until 2030 combining different components of the two strategies were defined, and they were assessed using MCDA.

In Hickman et al. [60], transport futures were examined by assessing the sustainability impacts of different transport policy trajectories. Two scenarios with different packages of policy measures on a regional level in the UK were developed in an iterative process, assuring the final scenarios adhering to the set criteria. These were being in compliance with national and local sustainability policies, being deliverable and feasible and finally being in alignment with set sustainability criteria on a local level. The MCA assessment was combined with scenario testing in a participatory process.

The choice of electricity generation technologies was the focus in Streimikiene et al. [61], using MCDA for assessment of six different electricity generation technologies in Lithuania (nuclear, natural gas, bio CHP, geothermal, hydro and wind). The list of technologies and their features were defined and assessed by experts on different sustainability aspects.

Cost benefit analysis (CBA)

Cost Benefit Analysis (CBA) is an economic tool, which aims to assess total costs and benefits, including environmental costs, of a project [95]. By weighing the costs of different alternatives against their benefits, this tool can provide guidance as to which option to choose.

Baard et al. [34] suggest CBA as an optional tool for municipal decision-making for climate adaptation strategies along or instead of the goal conflict analysis earlier mentioned. If the results from the qualitative impact assessment described in the previous section on qualitative mapping of sustainability impacts proved too weak, i.e. if no decision could be made as to which scenario or alternative is preferable, then a CBA could help ranking different options. The CBA of the different technical and administrative measures was done back-office and only for one of the explorative scenario (worst case scenario with the most significant climate impact 2060) [34]. The benefits considered were both environmental (ecological benefits) and social (recreational benefits, freshwater supply to households etc.)

Combined scenario building and assessment methods

The methods discussed above are more or less stand-alone tools that can be used to assess scenarios. The scenario generation and the assessment can in these cases be seen as separate activities. In some cases however, the assessment of the scenarios is an integrated part of the scenario generation method and can therefore not be seen as an isolated exercise. Below are some examples discussed where some sort of sustainability assessment is integrated in the scenario modelling.

Systems modelling and simulation

Computer-based modelling and simulations of complex and dynamic systems can be applied for prediction, forecast, policy making, social learning, theory building, system understanding and experimentation [96]. A number of techniques can be used for systems modelling and simulation, e.g. system dynamics and agent-based modelling.

Systems Dynamics (SD) is a computer-based approach used for policy analysis and design. SD considers dynamic problems that arise in complex social, managerial, economic or ecological systems [97]. The basic principles of SD approaches are feedback control theory, understanding the decision-making process, and the use of computer-based technologies for developing simulation models [98]. There are two layers of SD – qualitative, at a systems thinking level, and quantitative, including a computer simulation model [99]. Agent-based (AB) modelling covers the interaction between autonomous entities in a system [96]. AB models usually have three components: a set of agents; a set of agent relationships and ways of interaction; the agents’ environment ([99] based on [100]).

SD simulation can be used as an add-on to for example Social Impacts Assessment as illustrated in Karami et al. [62]. The dynamics are defined based on collected data, for example from affected households, and then used to forecast future conditions and consequences.

An example of using SD modelling for sustainability assessment of scenarios is a study by Hilty et al. [63], where SD approach is used in combination with scenario techniques and expert consultations to assess the impact of ICT on environmental sustainability in the EU in 2020. The study was revisited by Ahmadi Achachlouei and Hilty [64]. The SD model was used both for generating scenarios as well as for calculating results for some sustainability indicators. Systems modelling approach can be used together with LCA for assessment of the environmental impacts and resources used [2], and it also has the potential to be used with other Environmental System Analysis tools [99].

Computable general equilibrium (CGE) models

Computable general equilibrium (CGE) model is a quantitative method for evaluating the impact of economic and policy shocks in the economy as a whole [101]. The approach allows reproducing the structure of the whole economy, including all existing economic transactions between all economic agents, e.g. industrial sectors, households, government, etc. The CGE approach is especially useful for evaluation of policy implementation with the expected complex effects materializing through different transmission channels.

The Environmental Medium Term Economic Model (EMEC) is a CGE model of the Swedish economy. The model was developed and used to analyse the interaction between the economy and the environment in a number of climate policy analyses in Sweden ([102,103,104]. EMEC is a static CGE model, which includes 26 industries and 33 composite commodities (e.g. agriculture, fishery, gas distribution, road passenger transport, etc.). Composite commodities for domestic use consider imports and exports. The groups considered in the model are: utility maximizing households, profit maximizing firms, public sector and a foreign sector. As results, EMEC have several socio-economic indicators, but also environmental indicators such as GHG emissions and waste amounts. Besides climate policy, EMEC has also been used for example to explore policy measures needed for non-increasing future waste quantities [105]. It was also combined with a waste model and LCA for evaluating scenarios of future waste management [65]. Besides generating scenarios for the whole economy, it also includes environmental sustainability related indicators such as emissions of CO2 and generation of waste, in addition to socio-economic indicators.

Integrated models used for IPCC scenarios modelling and assessment

A number of integrated models were used for modelling and assessing IPCC scenarios [66,67,68]. Each of these integrated models is a set of multidisciplinary models consisting of macroeconomic models (e.g. CGE models), energy systems models (which can be based on SD), land and water use models and GHG emissions models [106]. Based on exogenous drivers, the models can be used for calculating GHG emissions in different scenarios. As results they will also produce other sustainability related results such as other emissions and use of resources.

Tools for strategic environmental assessment

Strategic Environmental Assessment (SEA) is a strategic change-oriented procedural tool used for policies, plans and programmes [17, 20], which can include different analytical tools. There is however, no single method that is most suitable in all situations for identifying, describing and assessing all types of significant environmental effects for use in an SEA. Many tools are suggested in the literature [17,18,19,20,21,22, 107]. Most methods mentioned are included in our review above e.g. checklists or life cycle analyses. Additional methods listed that could potentially be useful for assessing scenarios are:

  • Geographical information system (GIS)

  • Overlay/constraint maps

  • Trend analyses, extrapolation

  • Vulnerability analyses

  • Risk assessments, risk analyses

  • Carrying capacity, Ecological footprint

  • Planetary boundaries

The SEA reports that have been reviewed in this project only constituted a limited sample. It is however, still interesting to note that the tools described in the SEA literature and recommended by the EPA are rarely used in practice. Semi-quantitative or qualitative assessments are most often used. Checklists based on regional environmental goals and overlay maps are the most common methods used to compare alternatives. Most assessments were based on experience and knowledge of the people doing the SEA. It is also interesting to note that the aspects analysed are different in the different reports. Most common are environmental aspects such as water, noise, landscape, natural environment, cultural environment, risk and air. Others include also social and economic aspects such as security, equality and integration, attractive location, and effective land use.

Discussion

Reflection on the papers reviewed

Seven of the reviewed papers were methodological papers or a combination of a methodological paper applied in a specific case [31, 44, 50, 58, 78, 87, 108], while the remaining articles were primarily case studies. Apart from the methodological papers, few of the reviewed articles clearly described the reasons for the choice of assessment method in the first place, although some discussed the benefits of having used a specific method. In most cases there was no reflection as to the purpose of the assessment. Instead, the choice of methods appeared to be done more or less ad hoc. This suggests that there is a need for further discussions of the appropriateness and usefulness of different sustainability assessment methods for scenarios. Practitioners need to reflect more on the choice of sustainability assessment methods and there can be a need for methodological development.

From the review, it seems clear, that there is not one single method that can be used in all cases. Several methods were often used in combination such as in Kowalski et al. [35] where LCA, MCA and a qualitative assessment of social aspects were performed.

In parallel to the choice of method, the choice of aspects to be included was often not motivated either. In the reviewed SEAs, even if relevant sustainability aspects were selected as part of the process, no explanation was provided regarding why some aspects were deemed relevant and others not. In some papers, the selection of aspects was done in a participatory process. In these cases, obviously, the values of the participating stakeholders formed the basis for the selection. Nearly in all papers and plans analysed, environmental aspects and climate as well as GHG emissions were the most common aspect considered but other environmental issues such as biodiversity or land use were also included in several cases.

About half of the reviewed articles and two SEAs of plans addressed social sustainability issues. However the aspects included varied quite substantially. In some articles only one aspect, e.g. employment was assessed whereas in some articles (e.g. [41]) up to ten social aspects were considered. Distributive issues were rarely addressed. Only seven papers and one municipality plan in our review mentioned issues of equity, social justice or impacts on vulnerable groups but even fewer further described e.g. which vulnerable groups were identified.

Zijp et al. [8: 222] claim that assessments in practice are often done by experts without involving stakeholders and with “poor question articulation”. Procedural tools, such as the assessment frameworks reviewed here may therefore be useful, as an initial step, irrespective of the type of scenarios. They may guide researchers and practitioners in the assessment, as to what its intended purpose is, which sustainability aspects to include or whether to include stakeholders in the assessment process. In this context, SAFS [31] has been specifically developed for the assessment of transformative long-term scenarios and might be particularly useful for our focus on scenario assessments.

Linking the assessment aim to types of scenarios

As has been suggested in the reviewed frameworks and elsewhere (e.g. [8]), reflecting about the purpose of the assessment might be a useful starting point as a first tool selection criteria. To some extent the purpose of the assessment may also result from the type of scenario in question. We attempt to illustrate this below by mapping which questions could be relevant for assessments to answer depending on the types of scenarios as suggested by Börjeson et al. [10] (see Fig. 1). The choice of assessment method may also depend on scenario type specificities such as the degree of change and time horizon. A further selection may then be needed considering the object of study (e.g. product, sector or the whole society (c.f. [2]), the type of output provided by the different methods or if a life-cycle perspective is wanted or not (c.f. [7, 8]. Some of these aspects are described below.

Fig. 1
figure 1

Examples of questions assessment could aim to answer depending on scenario type

Full size image

The different assessment methods may indeed be more or less suitable depending on the time horizon for the scenarios. It is noteworthy, however, that the time horizon chosen for the assessment was not specified in several papers. It will of course be more difficult to make precise quantitative analysis for longer time periods than shorter. But also the pace at which different sectors change is relevant in this context. For example, the ICT sector is changing rapidly. It is interesting to note that the quantitative scenario studies looking into ICT reviewed here [48, 63, 64] all had a fairly short time span, whereas Arushanyan et al. [31] having a longer time perspective chose a qualitative assessment. On the contrary, several case studies using e.g. LCA for assessing energy systems had fairly long-term timeframes, possibly indicating a slower pace of change in the environmental performance of different energy technologies. Another reason could be that alternative energy technologies are central to address long-term challenges such as climate change mitigation, which usually require an analysis over a longer time horizon.

Predictive scenarios

The purpose of assessments of predictive scenarios would typically be to address the question: "What will happen and what are the sustainability impacts”?

For both forecasts and what if scenarios, which often are rather short-term types of scenarios, analytical tools such as quantitative tools relying on databases (e.g. LCA and S-LCA) might be useful to assess social and environmental impacts. Striving for accuracy and conducting an uncertainty analysis should be important aspects here. Qualitative tools may however also be used to assess sustainability impacts or identify societal goal fulfilments, such as matrices.

However for both forecasts and what if scenarios consisting of external factors, i.e. factors beyond the control of the intended scenario user but deemed important to consider, no prioritisation of scenarios need to be made as there are respectively no distinct alternatives to choose from nor strategic decisions to be made for scenario users.

For more strategic what if scenarios, i.e. which illustrate different strategies and actions taken by the intended scenario user, however, a prioritisation of alternatives could be relevant and tools to aggregate impacts might be useful, e.g. MCA.

Explorative scenarios

Explorative scenarios consist of external scenarios and strategic scenarios [10]. Such scenarios are often medium to long-term and can imply profound changes compared to today’s situation.

According to van der Heijden [109: 6] "External scenarios play in the contextual environment". Because of the focus on external factors, external scenarios describe developments that are beyond the control of the intended users [110, p. 31] and rather serve as contextual inputs to assess strategic scenarios, functioning as a sort of uncertainty analysis. For such scenarios it is not relevant to choose an alternative. As an example, different IPCC scenarios related to different levels of global warming and societal developments, can be used as external scenarios for assessing strategic scenarios.

Explorative strategic scenarios, on the other hand, depict what could happen if the intended scenario users make alternative strategic decisions and resulting actions. Assessments of such scenarios may have to answer a variety of questions and different tools may be combined. Estimating what could happen and what the sustainability impacts of different alternatives could be is one of them. Comparing how such scenarios would fulfil specific societal sustainability goals is another one.

A simple extrapolation of past data may not fit the purpose of assessing long- term scenarios with a large degree of change and the use of quantitative assessments for long-term developments characterised by high uncertainty risk giving a false impression of certainty of the results [31], which is why Arushanyan et al. advocate for a qualitative assessment of environmental aspects within the SAFS framework tool [31]. Another approach can be to develop new data sets, which reflect possible external developments. For example, when it comes to substantial technological changes, Gibon et al. [108] developed an approach where LCA and IOA databases were adapted for a future scenario in line with a 2-degree global warming. Such a scenario implies radical technological changes in most industrial sectors. Using this database, different technologies can then be evaluated with the 2-degree scenario used as an external context for the assessment.

When assessing explorative scenarios, the level of accuracy and the importance of uncertainty analysis might be lower, due to the longer time horizon and the more profound degree of change assumed compared to the present situation. If the question is “what could the sustainability impacts be?” then the need for accuracy and precision is limited. The aim of explorative scenarios is often to develop a set of scenarios that spans a wide scope of possible developments. Data that are used in quantitative assessments could also be reflecting a wide scope of possible developments, so for example reflecting different external developments concerning technological development. The use of different scenarios can be seen as uncertainty assessment, limiting the need for other formalised uncertainty analysis.

As explorative strategic scenarios illustrate different strategic choices, another potential relevant question could be how to prioritise between the different scenarios illustrating distinct strategies. In such cases, it may be relevant to use methods to aggregate impacts.

Normative scenarios

Normative scenarios answer the question: How can a target, which we here name “primary target”, be reached? [10: 728]. They consist of two types depending on the degree of change needed to reach the target(s): either by some “adjustments to current situation” (preserving scenarios) or through profound changes when the current structure itself is deemed to hinder goal fulfilment (transformative scenarios) [10].

Preserving scenarios could be used by e.g. municipalities on how to reach goals in the short or mid-term within the existing structure. Apart from assessments for sustainability impacts and potential for alternatives to reach the chosen goal and other societal goals, some prioritisation between alternatives may be relevant.

Transformative scenarios are long term and imply very profound changes [10]. Backcasting scenarios with primary quantitative target(s) may require a quantitative or semi-quantitative assessment in order to assess whether this (or these) target(s) indeed is (are) fulfilled. This can be a challenge, since quantitative assessments of long-term scenarios with large changes are typically difficult to conduct. “Ready to use” models might not be flexible enough to reflect such large changes.

A tailored quantitative assessment, e.g. based on a spreadsheet model such as in Francart [111], might be used instead. One advantage of quantitative assessment is that it might make it clearer that the current scenario description may not be radical enough to achieve ambitious quantitative targets and the scenario descriptions may have to be revised so that the narratives are consistent with the results of the assessment, thereby contributing to scenario building.

Another advantage of tailored quantitative models, might be to highlight that the changes needed to reach e.g. climate goal may be due to external factors and not to the strategies explored in scenarios. A quantitative assessment may help identify the factors that are crucial for reaching the primary target.

So the aim of such tailored quantitative assessments may be to better communicate an order of magnitude of the changes needed, a clearer differentiation of the impacts of different scenarios and to highlight which factors may contribute most to a lessen impact, rather than to provide an exact assessment of goal fulfilment. This should be made clear when communicating the assessment results in order to avoid giving a false sense of accurateness and certainty.

Qualitative assessments may be used to assess impacts on other sustainability criteria. A qualitative goal conflict analysis could for example be suitable. Even when some aspects are assessed as a result of the scenario analysis, many sustainability aspects are typically left uncovered in the scenario development. This, in turn, could lead to a sub-optimisation of the policy measures chosen to address the challenges identified during the scenario process [12].

The assessment of future scenarios can be an area where researchers from different disciplines have to meet. Arushanyan et al. [31] highlight the discrepancy between what researchers assessing scenarios and futures studies researchers see as the purpose of the scenario assessment. For the first ones, especially when conducting quantitative assessments, the scenarios should give as many and as precise details about the aspects that influence what is to be assessed. For the latter on the other hand, scenarios should sometimes avoid being too precise as uncertainty is great and they see the assessment more as a basis for discussion. According to Robinson [12: 838], "it may also be the case, given the level of uncertainty inherent in future analysis, that qualitative impacts (the "feel” of the scenarios, illustrated by rich textual descriptions) may be as, or even more, meaningful than estimates of quantitative impacts.

Although in principle one could ask which of different backcasting scenarios should be prioritised or chosen, this may be a less relevant question. For such scenarios, assuming long-term horizons and profound changes, the aim might rather be to highlight important issues that need to be taken into account in current policies rather than “selecting” a specific desirable future. For example, Dreborg [110] suggests that instead of choosing between futures, to consider development paths and related images of the future as examples of developments to unveil which strategic issues may be encountered towards goal fulfilment. However, methods for aggregating impacts can help identify features in the scenarios that could be determinant for the performance on other sustainability criteria such as the importance of the level of energy demand unveiled with MCA in Anderson’s case study [57].

As we earlier mentioned, the importance of considering both environmental and social aspects has been highlighted [9]. The need to integrate both aspects and to account for their interrelationship has also been discussed [112]. A combination of several tools might therefore be needed. Ideally assessments of normative scenarios (backcasting) should be done in an iterative way so adjustments can be made to scenarios if backcasting goals are not fulfilled in a first assessment and if additional sustainability issues are identified and could be addressed [12, 36]. The SAFS assessment framework [31] suggests in this context to reiterate the assessment of social impacts of scenarios by considering the resulting environmental impacts of scenarios as contextual factors in the second round of assessment e.g. what an increased use of minerals might mean for social aspects such as health.

Further reflections on sustainability assessments of future scenarios

As stated above, distributional issues were rarely considered. Walker et al. [113] claim that one reason why negative environmental impacts are unequally distributed could be that distributional issues have not been properly addressed in methods for assessing and evaluating policies in the UK. Munda [114: 307]) argues that "any policy option always implies winners and losers" and it is thus imperative to check if a policy option seems preferable just because some dimension (e.g. the environmental) or some social actors (e.g. the low income groups) has not been taken into account. According to Svenfelt et al. [36] there is therefore a need to consider other types of assessment for scenarios than environmental ones, such as social impact assessment, gender impact assessment or equality impact assessment.

Regarding municipal plans, Walker [115: 312] argues that there is no systematic analysis of distributional inequalities in impact assessment methods in the UK and that such an "analysis of the social patterning of impacts and benefits from projects, plans and proposals" along with a focus on procedural justice issues could be made. Some tools have been developed to encompass distributional issues such as CBA, however, Munda [114] notes that even if distributional issues have been included in CBA, it has been more in theory than in practice.

The analysis of how a decision would impact on current society groups may also be relevant to do considering future generations. According to Gasparatos et al. [116] sustainability assessments ought to include issues of intergenerational and intragenerational equity. Such analytical tools may therefore be needed or existing tools should be further developed and used in practice in a more systematic way.

The need to consider also second-order effects in the assessment was already highlighted by Robinson [12]. There are a number of different types of second order effects including direct and indirect economic rebound effects and time rebound effects (e.g. [117]). Different types of assessment methods can capture some of them. For example, CGE-models and IOA can capture some types of economic rebound effects [117]. The systems dynamics model by Ahmadi Achachlouei and Hilty [64] included some time rebound effect. Taking such effects into account might change the outcome of sustainability assessments in a substantial way, and they should therefore be considered in future assessments.

Further, although our review is not comprehensive, we found strikingly few papers dealing with assessment of scenarios, suggesting that Robinson’s concern with a neglect of that last step might still be valid. This was also illustrated recently at the Futures Conference For a Complex World held in Turku, Finland, in June 2017, where the issue of scenario assessment was not prominent in the conference programme as very few presentations touched upon the subject. This is an issue that therefore needs further attention and our paper can be seen as a contribution to such a discussion.