Contents

1 Introduction

Assessment of the resilience of farming systems has received increasing attention over the past decade. The concept of resilience was first introduced for ecological systems by Holling (1973) and later applied to social-ecological systems by Gunderson et al. (1995). Farming systems are social-ecological systems. These systems comprise biophysical, technical, and social elements that are subject to disturbances by both external factors (such as market fluctuations) and internal factors (such as pests and diseases) (Walker et al. 2006). A farming system is defined as “a population of individual farm systems that have broadly similar resource bases, enterprise patterns, household livelihoods, and constraints, and for which similar development strategies and interventions would be appropriate” (Dixon et al. 2001). The concept of resilience describes how farming systems cope with different perturbations, be they noise, shocks, stresses, cycles, or trends (Urruty et al. 2016). Many users of the resilience concept—such as academics and development actors who strive to strengthen resilience of vulnerable groups and systems—view it as a trait that systems need to hold to deal with the turbulent environments they operate in. Other authors have pointed to the “dark sides” of resilience: poor people may resourcefully withstand and cope with adversities, yet they remain marginalized by the status quo of the socio-economic context they live in (Berkhaut 2008; Wedawatta et al. 2010). When a system is in such an undesirable state, resilience may be problematic, as it maintains broader lock-ins and hinders transitions (Oliver et al. 2018). In such a case, resilience needs to be lowered—to enable the system to change—and then be strengthened again, once the system is in a desirable state (Holling 2001).

With the notion of resilience being increasingly applied to farming systems, there is general agreement that theory should be supported by empirical methodologies that facilitate assessment of resilience (Serfilippi and Ramnath 2018). Assessment of the resilience of farming systems involves identifying how resilience is created, maintained, and diminished (Quinlan et al. 2016). Walker et al. (2004) defined resilience as the capacity of a system to absorb perturbations or disturbances and reorganize while undergoing change, so as to still retain essentially the same function, structure, identity, and feedbacks. While this definition is widely accepted, a variety of assessment approaches seem to be based on different theoretical positions in terms of the combination of key concepts used and the explanation of the relationships among these concepts. Over the past decade, multiple authors have published resilience assessment frameworks (see overviews in Meuwissen et al. (2019); Serfilippi and Ramnath (2018); Fang et al. (2018)). This theoretical heterogeneity is reflected in a variety of resilience assessment approaches, focusing on different elements, in particular on (1) whether resilience is a capacity, a process, or an outcome; (2) what the causal relationships are between resilience and other key concepts, such as adaptive capacity, vulnerability, and stability (Meuwissen et al. 2019; Urruty et al. 2016); and (3) whether it adds value to distinguish between multiple capacities such as absorptive, adaptive, and transformative capacities (Béné 2013).

Serfilippi and Ramnath (2018) and Nikinmaa et al. (2020) also showed theoretical and methodological heterogeneity in the assessment of resilience in other fields, respectively, in disaster management and forest sciences. The lack of agreement on the resilience concept and on its elements contributes to misunderstandings in the assessment of resilience and of the impact of interventions aimed at strengthening it (Béné et al. 2014; Brown 2014; Córdoba Vargas et al. 2019; Meuwissen et al. 2019). Depending on the theoretical position and assessment approach chosen, a farming system may be evaluated to be more or less resilient. This poses challenges for a shared understanding and for taking action to enhance desirable farming system resilience, particularly when comparing findings and outcomes within and across systems. Hence, the debate on resilience assessment will benefit from increased clarity, both conceptual and operational.

Because of the challenges stemming from this methodological and theoretical heterogeneity, this paper aims to systematically review and analyze the literature on resilience assessment of farming systems and to identify patterns in theoretical underpinnings that reflect the variation in resilience assessment approaches. Hence, we review the existing literature to (1) shed light on the theoretical positions that have been advanced in literature for assessing resilience of farming systems; and (2) identify distinctive merits in the different approaches used to assess farming system resilience.

2 Review protocol

This systematic literature review was based on the recommendations by Petticrew and Roberts (2006). The steps followed are outlined in Figure 1.

Fig. 1
figure 1

Steps used in this review with resulting numbers of papers selected. Authors’ elaboration based on Petticrew and Roberts (2006). Lenses: T–traditional, V–vulnerability, C–capacities, A–agroecology, O–others.

Full size image

2.1 Review question

The research aimed to discover how the assessment of resilience of farming systems has been conceptualized and operationalized and what this implies for further development of assessment approaches. This topic was broken down into three research questions: (1) What theoretical underpinning can be identified in the approaches used to assess the resilience of farming systems? (2) What methodological characteristics can be identified? and (3) What are the likely implications of the theoretical and methodological findings on resilience assessment?

2.2 Inclusion and exclusion criteria

Selection criteria for publications were established, i.e., English language, peer reviewed, focusing on resilience assessment in agri-food systems. A restrictive search string was developed that included the different scales of systems involved in farming—production activity, farm, farming system, and farming as part of the supply or value chain—as well as the different terms used to assess the resilience of such systems (measure or assess): (((farm* AND resilience) OR (chain AND resilience)) AND ((resilience AND measur*) OR (resilience AND assess*))), alternatively written as ((farm* OR chain) AND resilience AND (measur* OR assess*)).

2.3 Literature search for relevant studies

In step 3, a search was run on the core collection of Web of Science. This database was chosen because it indexes interdisciplinary studies, while its coverage of records is comparable to that of Scopus. The first run in August 2018 yielded 798 papers, and another run in early April 2019 (covering August 2018–April 2019) yielded 176 papers. An additional 42 papers were added from Web of Science database alerts until the end of April 2019, bringing the total to 1,016 papers.

2.4 Selection of relevant studies

To select the relevant papers, in step 4, the first three authors reviewed the 1,016 papers to remove the irrelevant papers. A sample of the papers was reviewed by either two or all three authors, as were papers for which the relevance was not entirely clear. The remainder of the papers were divided between the three authors and reviewed by one author. Papers were considered irrelevant if any of the following exclusion criteria applied:

  1. 1.

    Paper did not focus on agri-food systems

  2. 2.

    Paper focused on resilience of rural communities or of networks without explicitly talking about any form or scale level of farming or agri-food system

  3. 3.

    Paper did not focus on assessment of resilience; resilience and its assessment were mentioned just in passing, without playing a significant role

  4. 4.

    Paper in which only the abstract was in English

  5. 5.

    Paper was published in a journal not meeting peer review criteria described in UlrichsWeb (UrlichsWeb 2021), a global series directory with detailed information on 300,000 journals and other periodicals; if information was not available in this directory, we consulted the journal website to verify the information on peer review; peer-reviewed conference materials were retained

  6. 6.

    Paper was published before 2010 (these were very few)

After all irrelevant papers were removed, 123 papers remained for further appraisal.

2.5 Appraisal of selected papers

In step 5, we used the following procedure:

  1. 1.

    Code selection and conceptual framework—We looked for patterns in the theoretical and methodological characteristics of approaches to assessing resilience of farming systems. Based on a first reading of the 123 selected papers, we inductively developed a coding frame (see Table 1), which contained the following conceptual elements:

    • The codes “farming system scale” and “main functions of system” (“of what”) and “perturbations being considered” (“to what”), following the recommendation of Carpenter et al. (2001) to ask “resilience of what to what?” (see Table 1 for examples)

    • Theoretical position taken by the authors (or the “lens” through which they look at resilience), following Zawacki-Richter et al. (2020)

    • Outcome definition of resilience—what does resilience lead to?

    • Factors that contribute to resilience, supporting a system in responding to perturbations, including:

      • Adaptive capacity—the capacity to anticipate perturbation (risk); to design and implement strategies so as not to be harmed by those perturbations; and to maintain system function, structure, identity, and feedbacks

      • Absorptive or buffering capacity—the capacity to moderate or buffer the impacts of shocks on livelihoods and basic needs

      • Transformative capacity—the capacity to create a fundamentally new system when conditions make the existing system untenable (Béné et al. 2012)

      • Practices—agroecological and management activities for farming and marketing

      • Resources—tangible and intangible assets, including natural, economic, physical, human, and social resources (DFID 1999)

    • Dimensions of resilience—the system aspects that may be affected by perturbations, such as environmental, economic, social, and technical

    • Scoring method used to qualify or quantify level of resilience

    • Number and types of indicators used in scoring level of resilience.

  2. 2.

    Coding—We then systematically coded selected papers; codes and sub-codes were summarized in MS Excel. Ambiguous cases were discussed in the team.

  3. 3.

    Reiterations of code selection and coding—After assessing about 25% of the papers, we refined the coding frame based on additional insights obtained; papers already assessed were assessed again, but by different team members, and the remainder of the papers was coded. Papers for which a code was less obvious or was unclear were discussed between the three first authors until agreement on the most suitable code was reached.

  4. 4.

    Final definition of sub-codes—After the assessment, a number of sub-codes were combined in more general sub-codes, as the number of papers for some original sub-codes was deemed too small (such as “system scale” and “perturbations,” Table 1).

Table 1 Codes and sub-codes used to characterize papers (portraying only the codes that show clear patterns).
Full size table

2.6 Synthesizing study results

The code “theoretical position” (or “lens”) was selected as grouping parameter for further analysis. The four identified lenses (excluding “O–others”) were thus used as ordering categories for the next section. Frequency scores per lens were calculated for the codes in Table 1. These were turned into histograms that illustrate the variation across sub-codes between the lenses. Display was either by number of papers or by share of papers. Additional details from the papers—on the processes and practices through which adaptation, absorption, and transformation were achieved—were used to enrich and triangulate the findings, including information on the specific crops, biophysical conditions, and use of resources.

3 Results and discussion

3.1 Four lenses, four theoretical positions

Traditional, vulnerability, capacity, and agroecology lenses

The four lenses identified (i.e., T–traditional, V–vulnerability, C–capacities, A–agroecology) were used for further clustering and analyzing. Based on the papers in each lens, we first describe the characteristics of these lenses. Figure 2 displays the emphasis that each lens puts on the three different and complementary foci of analysis: the outcome definition that papers using this lens imply about resilience (stability, transformation, and/or reduced vulnerability), the prominent factors contributing to resilience (capacities, practices, and/or resources), and the perturbations the farming system is exposed to. We offer this as a novel causality framework for understanding how different lenses view resilience.

Fig. 2
figure 2

The causality framework showing prevalent elements in resilience assessment of farming system in four lenses (authors’ elaboration). N.B. Red-brown coloring denotes item receiving major emphasis in assessment approach; black denotes intermediate emphasis; light gray denotes limited or no attention.

Full size image

The Traditional lens (T, 39 papers) can be regarded as representing the foundational school of theory on resilience of social-ecological systems, on which other lenses build. The papers use the main theory and approaches developed over the past four decades. Key concepts were summarized by Walker et al. (2006) as consisting of adaptive cycle, panarchy, resilience, adaptability, and transformability; four authors of this paper, Walker, Gunderson, Folke, and Carpenter, are each quoted in ~75% of the papers using this lens. These prominent authors repeatedly indicate the need to define approaches and metrics to assess resilience. Resources and adaptive capacity are regarded as contributing factors that help social-ecological systems to retain their function in the face of perturbations, with stability as the outcome of resilience. Some attention is paid to transformation as a possible outcome. For example, paper T34 (Tittonell 2014) (see Supplementary Material for details on reviewed papers) connects regime shift to the three future options smallholders have, described by Dorward et al. (2009) as “hanging in,” “stepping up,” or “stepping out.”

The Vulnerability lens (V, 43 papers) looks mainly at adaptive capacity from the viewpoint of the vulnerability framework as described by the Intergovernmental Panel on Climate Change (IPCC 2014). Systems that are easily exposed and highly sensitive to shocks can be said to be vulnerable to perturbations (V03, Alayon-Gamboa and Ku-Vera 2011). Vulnerability is reduced by the system’s adaptive capacity to deal with these shocks, for which resources and practice changes are important (V03). The definition of vulnerability in Adger (2006) is “the state of susceptibility to harm from exposure to stresses associated with environmental and social change and from the absence of capacity to adapt.” In its focus on adaptive capacity, this lens follows the IPCC definition of adaptive capacity (Allwood et al. 2015): “the ability of systems, institutions, humans, and other organisms to adjust to potential damage, to take advantage of opportunities, or to respond to consequences” (p. 1251).

The Capacities lens (C, 24 papers) builds on the T-lens and particularly on the work of Walker et al. (2004) and Osbahr et al. (2010). Béné et al. (2012) focus on two capacities apart from adaptive capacity (also common in T- and V-lens): absorptive capacity, which is the capacity of individuals, households, and/or communities to moderate the impacts of shocks on their livelihoods; and transformative capacity, which is the capacity to create a fundamentally new system when shocks in ecological, economic, or social structures make the existing system untenable. Most of the papers using the C-lens work with these three capacities (absorptive, adaptive, and transformative), but some papers use “the capacity for learning and adaptation” or “the capacity to self-organize” instead of adaptive or transformative capacity (i.e., C05, Galappaththi et al. 2017; C10, Jacobi et al. 2018; C18, Speranza 2013; C19, Speranza et al. 2014). Papers to some extent elaborate on the type of perturbations and the resources used in dealing with them (e.g., C14, Shadbolt and Olubode-Awosola 2016) and give no or limited attention to practices.

The Agroecology lens (A, 9 papers) was used by papers that study farming systems from an agroecological perspective. Agroecology was defined by Dalgaard et al. (2003) as “the study of the interactions between plants, animals, humans and the environment within agricultural systems.” The focus in these papers on resources as determinant of resilience is mostly on nutrient flows (A03, Dendoncker et al. 2018; A05, Stark et al. 2018; A07, Vanegas et al. 2018). The papers focus on two core concepts, diversity and redundancy, which together are considered to be conditions for ensuring adaptability (Darnhofer et al. 2010 and paper A08 (Veisi et al. 2013)). Diversity means the variety in agricultural practices and resources used, including diversity of crops and livestock (A01, Bahadur et al. 2016) and of flora, fauna, and ecological functions (A03, Dendoncker et al. 2018; A06, Valencia et al. 2019). Diversity is seen as an important condition for redundancy, i.e., the extent to which elements are substitutable in the event of disruption or degradation (Norris et al. 2008). In most papers, these perturbations are not specified. Redundancy is considered to be an important contributor to stability, which seems to be the prevailing but often poorly articulated desired system outcome of A-lens papers.

Other (O, 8 papers)—Eight papers used another lens or an approach that did not fit in one of the four lenses. Approaches used were the Lifecycle Assessment for Climate Smart Agriculture (O01, Acosta-Alba et al. 2019), the MESMIS agroecosystem sustainability evaluation framework (O02, Astier et al. 2011), profitability analysis (O04, Hamerlinck et al. 2014), the Household Livelihood Resilience Approach (O05, Quandt 2018), and resilience being used as an indicator in sustainability assessment (O06, Valenti 2018). One paper used grounded theory (O07, Panpakdee and Limnirankul 2017) and two papers in this category did not contain enough information about their theoretical background to categorize them into any of the four lenses (O03, Bardsley and Bardsley 2014; R-O01, Alroe et al. 2017). Papers using other lenses have been excluded from analyses portrayed in Figures 4, 5, 6, and 7.

From here on, we display the four lenses in alphabetical order, as is done in Figure 2.

Literature emergence and growth across the four lenses

Out of the 123 selected papers, 16 were conceptual and review papers that did not describe any clear and implementable assessment approach, leaving 107 empirical papers for full appraisal regarding the research questions of this review (Figure 1). The Supplementary Material provides an overview of all papers assessed. Twenty papers portrayed elements of two lenses (i.e., 15 empirical and five conceptual and review papers, e.g., A01, Bahadur et al. 2016; R-C04, Cabell and Oelofse 2012). These were classified in the lens that best fitted their theoretical underpinning. All 123 papers were included in the description of lenses, paper metrics, and qualitative assessment (Sections 3.1 and 3.2), but for the more quantitative analyses of assessment approaches (Section 3.3), the conceptual and review papers and the papers using other lenses were excluded and analysis focused on the remaining 100 empirical papers grouped under the four lenses.

Numbers of papers published one in all of 2010 to a provisional peak of four per month in 2019 (Figure 3), although 2019 data were for the first 4 months only. Papers using the V- and T-lenses were published in significant numbers throughout the decade. The A-lens and C-lens started later, supporting the impression that they emerged in reaction to the T-lens and V-lens.

Fig. 3
figure 3

Number of papers published per month for the different lenses over the period January 2010 to April 2019. Lenses: A–agroecology; C–capacities; T–traditional; V–vulnerability; O–other.

Full size image

Prominent features in conceptual and review papers

Nine out of 16 conceptual and review papers elaborated conceptual issues from a theoretical angle (e.g., R-C05, Carpenter et al. 2012; R-T04, ten Napel et al. 2011). The others reviewed literature, or combined a review with conceptual elaborations (R-C01, Aboah et al. 2019; R-C02, Béné et al. 2014; R-O01, Alroe et al. 2017; R-T03, Quinlan et al. 2016; R-V01, Bailey and Buck 2016; R-V03, Elias et al. 2018; R-V04, Hansen et al. 2019). The objectives of the papers differed widely. Some papers offered recommendations for research or for application of the resilience concept (R-O01, Alroe et al. 2017; R-T01, Macfadyen et al. 2015; R-T05, Darnhofer et al. 2016; R-V04, Hansen et al. 2019). Others proposed a resilience assessment framework or indicator framework (R-C01, Aboah et al. 2019; R-C04, Cabell and Oelofse 2012; R-T02, Peterson et al. 2018; R-T04, ten Napel et al. 2011; R-V01, Bailey and Buck 2016; R-V05, Vroegindewey and Hodbod 2018). Some papers studied how to build resilience (R-T01, Macfadyen et al. 2015), reviewed resilience aspects of regional agri-food systems (R-V02, Brzezina et al. 2016; R-V03, Elias et al. 2018), or investigated the advantages and limitations of using resilience in the development field (R-C02, Béné et al. 2014; R-C03, Béné et al. 2015). The C-, T-, and V-lenses each contain five conceptual and review papers, while one review paper used an ethics perspective that did not fit in any of the four lenses (R-O01, Alroe et al. 2017). Important findings, such as the choice between specific and general resilience, have been used in discussions in this paper.

3.2 Complementarity between concepts used in the four lenses

The different theoretical positions lead to significant differences in further conceptualization. In this section, we present and discuss the differences, which are summarized in Table 2. Listed differences in assessment approaches are discussed in Section 3.3.

Table 2 Comparison of conceptualization and assessment approaches across four lenses (A–agroecology, C–capacities, T–traditional, V–vulnerability), with ✓ indicating “significant attention” and ~ indicating “some attention.”
Full size table

Outcomes of resilience

Resilience outcomes identified in this review were stability, transformation, and reduced vulnerability of a system. Due to the very definition of resilience (e.g., Walker et al. 2004) including the concepts of absorbing disturbances while retaining essentially the same function, most authors define resilience outcomes in terms of stability. C-lens papers also focus on adaptation and transformation, while a number of A-lens papers appear to see diversity as both means and outcome. The V-lens rather focuses on reduced vulnerability, which is quite specific to this lens. Urruty et al. (2016) point out that the concept of vulnerability originated from social sciences, being used for people, social systems, and countries; only in the past two decades was this extended to social-ecological systems such as farming systems. Resilience was originally used in engineering and ecology, before being applied to farming and other social-ecological systems. These origins have a bearing on the application of the concepts. According to Urruty et al. (2016), vulnerability focuses on the direct impact of specific perturbations on a system, while resilience is actually most relevant in the long term, to describe and understand system recovery processes (stability) or transformation into a different system state. Thus, the resilience concept focuses on the consequences of one to several perturbations, potentially including unpredictable ones, for the overall trajectory of the system. From this review, we get the strong impression that most “resilience assessments” using the V-lens are focusing only on the vulnerability aspect rather than on the broader resilience aspects. While these studies benefit from the broad palette of methodologies developed for vulnerability assessment (e.g., Barsley et al. 2013), the focus on vulnerability in V-lens papers potentially restricts the scope of resilience assessment.

Factors contributing to resilience

Proceeding with the 100 empirical papers in the four identified lenses, it became apparent that different lenses lead to different choices for the assessment of the factors contributing to resilience, be they capacities, resources, or practices (V24, Meldrum et al. 2018). Attention for capacities was particularly low in papers using the A-lens, which focused more on practices (Figure 4a). More focus on capacities would lead to deeper analysis about whether practices do actually lead to more resilience. Assessment of resources received attention in about two-thirds of all 100 papers, without strong differences between lenses. Apart from these three common contributing factors, four papers in the T- and V-lenses used other properties, such as attitudes and strategies toward risks (T28, Phuong et al. 2018; V10, Buelow and Cradock-Henry 2018) or performance of the system (T13, Fall et al. 2018; V12, Chodur et al. 2018). The latter appears to confuse contributing factors with resilience outcomes.

Fig. 4
figure 4

Contributing factors of resilience (a) and capacity types (b). N.B. n < 100 as papers not mentioning contributing factors/capacities were not taken into account. Lenses: A–agroecology, C–capacities, T–traditional, V–vulnerability.

Full size image

The distinction between multiple capacities was most prominent in papers using the C-lens (Figure 4b). All papers using the C-lens used adaptive capacity; 95% used absorptive and 86% used transformative capacity. A range of other terms was used to distinguish capacities (in 36 papers). These included coping capacity (e.g., C04, d’Errico and Di Giuseppe 2018; V32, Sieber et al. 2015) and capacity to learn (e.g., C18, Speranza 2013; V29, Nguyen et al. 2018). Sometimes these terms were used alongside adaptive capacity, adaptability, transformative capacity, transformational capacity, or transformability, and sometimes they were used as synonyms.

Use of capacities varied significantly across lenses. The use of multiple capacities, prominent in the C-lens, was little developed in other lenses, particularly in the V-lens (exceptions include V28, Mutabazi et al. 2015). Specifically, the C-lens’ distinction of absorptive capacity was little reflected in other lenses, despite its prominence in Walker et al. (2004). However, it may be implicitly included in the V-lens’ use of “sensitivity to shocks and stressors.” A question deserving further analysis is whether low sensitivity is actually a result of high absorptive capacity or is caused by other relationships.

The relationship between absorptive capacity and robustness particularly remains conceptually contested. Across literature, robustness is regarded as a trait of technical rather than social-ecological systems and, according to Urruty et al. (2016, p. 15), represents “the complex interactions between the biotechnical factors of agricultural systems and external drivers of change.” Two C-lens papers (C03, Cochrane and Cafer 2018; C09, Jacobi et al. 2015) follow this pattern and see absorptive capacity contributing to both robustness and resilience. Two T-lens papers (R-T01, Macfadyen et al. 2015; R-T04, ten Napel et al. 2011), however, prefer to use robustness instead of absorptive capacity and see robustness as contributing to resilience, a position that was also taken by Meuwissen et al. (2019). Further research could shed more light on the relationship between absorptive capacity, robustness, and resilience.

Walker et al. (2004) distinguished transformability from resilience, referring to adaptive rather than to transformative capacity when defining “capacity to reorganize.” The C-lens diverts from this position by distinguishing transformative capacity as one of three resilience capacities, thus including a system’s ability to transform—with changes in functions, structure, identity, and feedbacks—in its resilience. This may reverse evaluation of resilience in cases where continued absorption of and adaptation to shocks and stresses (a conservation orientation) may be considered less desirable than transformation to a different state. As various papers by Béné et al. show (e.g., R-C02, Béné et al. 2014; R-C03, Béné et al. 2015), this distinction of capacities—which can be built or strengthened—fits well with application in the development field. Despite the conceptual attention for transformative capacity, its operationalization receives little attention in C-lens papers, while some papers using other lenses (T21, Marshall et al. 2012; V23, Marshall et al. 2014a) do make such an attempt.

Perturbations

What we characterize here as a perturbation is described in the literature under a range of terms: hazard, threat, risk, disturbance, shock, and stress. Papers using the V-lens were most explicit about the relationship between perturbation and system response; whether an external shock or stressor results in actual perturbation in the system depends on the system’s exposure and sensitivity to the shock or stressor (e.g., V01, Abdul-Razak and Kruse 2017). As a conceptual example, chickens may be very sensitive to Newcastle disease, but exposure may be reduced by a barn with good biosecurity, while sensitivity may be reduced by vaccination.

Despite the V-lens’ explicit conceptualization of vulnerability to shocks and stresses, the strong focus on climate change results in omission of other shocks or stresses, such as market fluctuations or land scarcity, that in a specific study context may be at least as dominant as climate change. In other words, the resilience to climate change in papers using the V-lens is so specific that their well-defined methodologies are not easily extended to assessment of resilience to other perturbations. Assessment of multiple perturbations, or selection of the most pressing one, would do justice to the large variation between situations that farms find themselves in. Implications for operationalization are outlined below in Section 3.3. This touches on the debate about whether specific (or specified) resilience to one or a few perturbations may need to be contrasted with general resilience, i.e., the capacity of systems to adapt or transform in response to unfamiliar, unexpected, and extreme shocks (R-C05, Carpenter et al. 2012). Assessment approaches for general resilience tend to be rather different than for specific resilience, with more attention for relations and processes in a system (Darnhofer 2021).

Opportunities for complementary use

In response to the first research question, this review identified distinctive theoretical features of the heterogeneous approaches used to assess resilience of farming systems. Resilience appears to be viewed in three ways: as the outcome of a change process in which a farming system responds to shocks and stresses; as the capacity of a farming system to respond to shocks and stresses; or as that process itself, i.e., the complex interaction between perturbations and contributing factors. A general limitation of the entire set of literature is that each lens only covers certain elements of the whole resilience concept. This makes comparison of assessment approaches rather difficult. As listed in Table 2, particular limitations of the four lenses are that (1) high variation in approaches in T-lens papers offers little guidance on assessment methodology; (6) V-lens papers pair relative consistency in assessment approaches with a wide variety of conceptual foundations, focus on adaptive capacity as the only capacity contributing factor, focus on climate change at the expense of other perturbations, and often appear to assess vulnerability rather than resilience; (3) C-lens papers focus on capacities at the expense of attention for other contributing factors and perturbations; and (4) resilience in A-lens papers is poorly conceptualized. This gap was also noticed by Tittonell (2020), who, in an attempt to address it, proposed ten criteria to assess the contribution of any type of transition to building resilience and adaptability in agroecosystems.

Comparison of the conceptual elements used in the four lenses further raises questions about the causal relationships between them. Many papers are unclear about the perceived relationships between perturbations, contributing factors, and outcomes. They often equate resilience as a system trait with its contributing factors or with its outcomes. The novel causality framework we offer in this paper (Figure 2) may be useful for uncovering the theoretical underpinnings encountered.

One conceptual issue that remains unresolved is what relationships exist between the five factors identified as contributing to resilience. In a recent paper, Meuwissen et al. (2019) suggest a distinction between “resilience capacities” (which they label “robustness,” “adaptability,” and “transformability”) and “resilience attributes” (including but not limited to practices and resources). The latter are seen as contributing to resilience capacities and as being grounded in the adaptive cycle processes of agricultural practices, farm demographics, governance, and risk management. Resilience attributes should be assessed in the context of diversity, modularity, openness, tightness of feedbacks, and system reserves—generic principles of strengthening resilience proposed by the Resilience Alliance (2010).

On the flip side, the respective resilience interpretations of the four identified lenses offer strong opportunities for complementary use, since each of them addresses a different section of the whole resilience concept (Table 2). The long-established T-lens offers theoretical underpinnings that are grounded in the core concepts identified by Walker et al. (2006). These authors describe patterns of abrupt change in terms of adaptive cycle and panarchy, which elucidate the dynamics of systems within and across scales. Moreover, resilience, adaptability, and transformability are considered as properties of social-ecological systems that drive these dynamics. Complementing the T-lens, the V-lens pays considerable attention to perturbations, particularly in relation to a system’s exposure and sensitivity to shocks and stresses. Furthermore, the C-lens distinguishes absorptive and transformative capacities from adaptive capacity, thereby clarifying that resilience is a multi-capacity trait of a system: when a system is confronted with a shock or stress, it may respond by absorbing the shock, adapting to it, or transforming to another system state. Last but not least, the A-lens sheds light on several aspects: resilience arises from observable practices; system diversity is an important indicator; and farming is a social-ecological system. New tools such as Tool for Agroecology Performance Evaluation (TAPE) (FAO 2019) may provide further insight into the theoretical underpinning of the A-lens. In TAPE, resilience is one of ten elements of agroecology. It is measured both qualitatively (e.g., resilience through diversity, social resilience of the community, environmental resilience of the territory, and the capacity of the system to adapt to climate change) and quantitatively (e.g., through the measurement of economic, social, and environmental performance) (Mottet et al. 2020).

3.3 Operationalization of resilience assessment varies between and within lenses

This section addresses the second research question, which asks about the methodological characteristics of the reviewed papers. We focus on system scales and functions, type and number of perturbations assessed, assessment dimensions, and degree of quantification.

System scales and functions

We first focused on the question “resilience of what (system) to what (perturbation)?” (Figure 2 and Table 2). Figure 5a shows that all lenses except the V-lens focused primarily on farming system scale (which also includes “agroecosystem,” “farm,” and “household livelihood”). Those using a V-lens focused equally on production activity (crop or livestock) and on farming system scale. Papers focusing on larger system scales (which include “value chain,” “food system,” “community livelihoods,” and multiple system scales) mostly used the C-lens. In terms of system functionality, livelihoods received the most attention and environmental services the least attention in papers across the C-, T-, and V-lenses. Most papers using the A-lens focused on food production (Figure 5b).

Fig. 5
figure 5

System scales (a) and functions (b). Lenses: A–agroecology, C–capacities, T–traditional, V–vulnerability.

Full size image

Approaches and lenses used in the reviewed papers in general seem to be rather indifferent to system scales, system functions, and resilience dimensions. Higher attention for livelihood functions in the C- and V-lenses may well be correlated to their propensity to be used in connection with humanitarian assistance and development initiatives, particularly in Africa. The relatively strong focus on the farming system scale (including farm, household livelihood, and agroecosystem) may be explained by the fact that the system boundaries of farming as social-ecological system are most clearly drawn at farm level, a biophysical unit, economic unit, and, as farm household, social unit. However, selection bias toward this scale level cannot be ruled out—the search string with farm* and chain* primarily yielded papers on the intermediate scale code.

Type and number of perturbations

The papers studied resilience to shocks related to a wide variety of perturbations (Figure 6), such as pests and diseases, droughts, abrupt price fluctuations, and new land policies. “Climate change” and “biophysical” risks were the most common perturbations mentioned. Papers using the V-lens addressed these perturbations in 37 of the 38 papers (97%) (i.e., all except V15, Falkowski, 2015) (Figure 6). Moreover, 33 papers (87%) using the V-lens mentioned “climate change,” of which 22 papers (58%) focused on resilience to climate change only. The other category of perturbation receiving significant attention was that of “market and supply chain disturbances,” such as price fluctuations (e.g., C15, Shadbolt et al. 2013). Nearly half of the nine papers using the A-lens did not specify any perturbation to the system, compared to a quarter of the 19 papers using the C-lens and even lower shares for papers using the T- or V-lenses. Papers using the T- or V-lens, on average, also mentioned the highest number of perturbations, followed by papers using the C-lens. Sixty-three percent of all 100 papers addressed a single perturbation; for the papers using A- or V-lenses, this always concerned “climate change,” while for the C- and T-lenses this concerned various perturbations, including “climate change,” “biophysical,” and “market and supply chain disturbances.” All lenses showed variation in the number of perturbations addressed, from one to four perturbations per paper (A09, James and Brown 2019; T09, Diserens et al. 2018; V12, Chodur et al. 2018; V30, Perez et al. 2015).

Fig. 6
figure 6

Perturbations mentioned. Lenses: A–agroecology, C–capacities, T–traditional, V–vulnerability.

Full size image

Assessment dimensions

In terms of dimensions of resilience (economic, social, environmental, and technical), attention for social resilience was highest in papers using the C-lens (95%) and lowest in papers using the A-lens (37%, e.g., A06, Valencia et al. 2019) (Figure 7a). The A-lens papers studied fewer dimensions (2.5 per paper) compared to the other lenses (3.2 per paper). The technical dimension received most attention in papers using the C- or V-lenses (e.g., V25, Mkonda et al. 2018). Attention for the environmental dimension differed least between the lenses. Generally speaking, papers across lenses address multiple dimensions, most so in the C- and V-lenses and least so in the A-lens, which emphasizes the environmental and technical dimensions.

Fig. 7
figure 7

Dimensions covered by assessments (a) and degree of quantification (b) (6 is highest). N.B. For b), n < 100 as papers not scoring parameters were not taken into account. Lenses: A–agroecology, C–capacities, T–traditional, V–vulnerability.

Full size image

Qualitative and quantitative assessment approaches

The different theoretical positions and choices in conceptual elements resulted in papers showing a wide array of approaches, even within lenses. In assessing the methodological choices, we focused on the degree of quantification and the number and type of indicators used.

Assessment approaches ranged from qualitative, opinion-oriented methods to quantitative, index-oriented methods. We looked at two characteristics of these methods: (1) the degree to which the assessment applied quantification in data collection and analysis; and (6) the types and number of indicators used. The degree of quantification was scored as 1 (low) when the assessment of system resilience depended on actors’ judgement, be it the perception of system actors such as farmers or the perception of the observer/researcher, without specification of indicators or scales (Figure 7b). Higher scores (up to 6, see Table 1) were used for increasing quantification of indicators (from “describing and scoring” to “measuring”), increasing use of statistical and mathematical analysis, and increasing inclination to consolidate the information from multiple indicators into one or more compound indices.

The degree of quantification showed clear variation between lenses. Papers using the V-lens are most likely to use a more quantitative approach, in which (proxy) indicators are identified, their values are evaluated, and usually indices are crafted and/or statistical analyses are performed. Papers using the A-lens also tend to follow this pattern, albeit to a lesser extent. Papers using the C- or T-lenses showed more duality between either measured indicators and further quantitative analysis or using more opinion-based scoring. The issue of measurability of resilience appears to be related to the chosen theoretical position: whether resilience can be measured by using (proxy) indicators (e.g., T13, Fall et al. 2018) or whether resilience is an emerging system property that cannot be observed objectively (R-T05, Darnhofer et al. 2016). This is especially an area in which the difficulties of operationalization of resilience assessment come to the fore, with a multitude of assessment approaches resulting that use indicators, opinions, surrogates, Likert scales, best proxies, or indices. Still, the degree of quantification used in the papers is not always easy to determine. Moreover, two papers from the same (first) author could score differently (e.g., C14, Shadbolt and Olubode-Awosola 2016 vs. C15, Shadbolt et al. 2013; and V22, Marshall et al. 2014b vs. V23, Marshall et al. 2014a).

Figure 7b shows the degree of quantification and Table 3 shows the number and types of indicators identified in the papers. These results show that papers using the V-lens on average received high scores for degree of quantification, used many indicators (highest was V09, Berry et al. 2011, with 126 measured indicators), and most often used constructed indicators and calculated indices. This more quantitative approach to resilience assessment tallies with the V-lens’ focus on vulnerability to perturbation. Papers using the T- and C-lenses tended to rely relatively more on perceptions and qualitative scoring; they also used fewer indicators than the V-lens. These papers appear to regard resilience as an emerging and volatile property that should be observed rather than measured. Papers using the A-lens were in between as to the degree of quantification.

Table 3 Number and types of indicators and indices, for lenses A–agroecology, C–capacities, T–traditional, V–vulnerability.
Full size table

3.4 A roadmap for complementary use of multiple resilience lenses

This final section addresses the third research question on implications of the above theoretical and methodological findings on actual resilience assessment. The results suggest that progress has been made toward operationalizing resilience, but that the need remains for clarity about the link between theoretical positions and methodological approaches. The three interpretations of resilience identified in Section 3.2—resilience as the outcome of a change process the farming system undergoes; as the capacity of a farming system to respond to shocks and stresses; or as that process itself, i.e., a complex interaction between perturbations and contributing factors—may be so fundamentally different that a unified assessment approach appears to be a pipe dream. Rather than striving for convergence, revealing and mapping the complementarity between these theoretical positions might result in a clearer clustering of methodological approaches.

This review showed that theoretical underpinnings affect assessment methodologies, highlighting distinctive applications for each of the four lenses but, more importantly, offering scope for complementarity. Strong articulation of system dynamics in the T-lens and the distinction of capacity types linked to resilience outcomes in the C-lens ideally are combined with the more developed assessment approaches of the V-lens, while the focus on practices and diversity in the A-lens helps in connecting theory and agronomic reality.

As none of the assessed papers evaluated the results of resilience assessment through application of different approaches (based on different lenses), it would presumptuous to conclude that such results would differ, but this would certainly be worth examining. A key focus area may be the inclusion or exclusion of system transformation as part of resilience. This distinction between conservative and transformative resilience—i.e., between “sticking to the old” or “transforming to the new”—has repercussions for the distinction between desirable and undesirable resilience outcomes.

Five key choices in design of resilience assessment

The comparison of lenses in this paper highlights the opportunities to complement their respective strengths. Their relative contribution depends on the objectives of a particular resilience assessment. This review identified the following key decisions to be considered in operationalizing a resilience assessment strategy, which at the same time may direct research on resilience assessment:

  1. 1.

    Choice of system traits—system type, system functions, and system scale—Clarification of these choices is important to make an assessment feasible, to allow for better replicability of studies, and to disentangle inter-scale resilience dynamics, i.e., improvements to the resilience of one system scale, such as a value chain, may have negative repercussions for resilience of another scale, such as smallholder farming. Fit of assessment approaches with specific system traits requires more study.

  2. 2.

    Identification of perturbation(s) to be considered—Assessment of resilience against multiple perturbations—or against the most important one—requires evaluation of the likelihood of those perturbations occurring. This requires detailed knowledge of the context under study and its stakeholder interests (R-C05, Carpenter et al. 2012) and implies a risk evaluation step before resilience assessment is conducted (Urruty et al. 2016). However, only a few of the reviewed papers give evidence of such identification and evaluation of perturbations (C09, Jacobi et al. 2015; C10, Jacobi et al. 2018). Stakeholder interviews by the first author of this paper indicate that exposure to shocks and stressors differs between and within regions (between farmers). Moreover, shocks and stressors that rank high in exposure may not necessarily rank as most threatening. Reasons may include that differences between farms expose particular farms more to particular shocks and stressors (such as market fluctuations for more commercial farms) and that a strong enabling environment reduces the sensitivity to certain shocks and stressors, e.g., good public veterinary services reducing the risk of epidemic diseases.

  3. 3.

    General or specific resilience—Moving beyond assessment of one or multiple known perturbations, assessment of general resilience against unexpected and unspecified perturbation appears to be an underdeveloped area, with only three papers in this review paying cursory attention to it (R-C05, Carpenter et al. 2012; R-T03, Quinlan et al. 2016; and T34, Tittonell 2014). Strengthening of general resilience may be essential for smallholder farmers in areas with unpredictable or unmanageable risks, considering their resource limitations for risk analysis and risk management (Darnhofer 2021). Such farmers understandably prioritize the reduction of variation in system performance over maximizing output, even if that results in low performance levels (Urruty et al. 2016). Development of an assessment approach for general resilience against unexpected and unspecified perturbations may warrant further research.

  4. 4.

    Selection of contributing factors—The factors to be considered are capacities, resources, practices, or, preferably, a combination of these. This review showed how this choice depends heavily on the lens used, that it has significant repercussions for the assessment approach used and influences the desirability of resilience.

  5. 5.

    Selection of resilience outcomes—As discussed above, reduction of vulnerability is a justifiable short-term objective, for which the V-lens offers the most established assessment approaches when it comes to climate change. Stability of system performance adds a longer term perspective, assessment of which will benefit from elements of multiple lenses. The third outcome, system transformation—needed to deal with prolonged stress, high risk probability, or dissatisfaction with system performance—has an even longer time horizon and actually underlies many agricultural development interventions. While the C-lens intends to address this outcome through its focus on multiple capacities including transformative capacity (R-C02, Béné et al. 2014), adequate assessment approaches are not yet developed by any of the lenses.

4 Conclusion

This paper has used a series of codes to analyze and assess 123 papers on resilience assessment of farming systems, in order to systematically review and analyze the literature and to identify patterns in theoretical underpinnings that reflect the variation in resilience assessment approaches. In order to guide evaluation of resilience of farming systems, the review focused on how resilience is conceptualized and operationalized. It pointed out that the four different lenses identified do offer a comprehensive but equivocal set of (causal) links between perturbations, factors contributing to resilience, and outcomes of resilience. This results in a novel causality framework that is then used to identify the complementarity of lenses in covering the whole resilience concept and in operationalizing resilience assessment. Conceptualization offers much complementarity of lenses in terms of resilience factors and causal links. Views of whether resilience is a system outcome, a system capability, or a change process are not well articulated in the assessed papers and may cause conceptual confusion and methodological deviations. Assessment approaches offer commonalities in terms of covering multiple system traits and assessment dimensions and offer complementarities in terms of quantitative vs. qualitative approaches and perturbation types covered. Assessment of resilience against unexpected, unfamiliar, and extreme shocks (i.e., general resilience) is an area that appears underdeveloped. With the different conceptualizations and assessment approaches chosen, decision-makers may evaluate a particular farming system to be more or less resilient, with implications for the design of interventions to enhance its resilience.

The analysis for the first time suggests that resilience assessment methods can be developed further by complementarily drawing from the strengths of the different perspectives. It identifies five key choices that need to be made in assessing resilience, each representing an area of further research. Specifically, more attention needs to be directed to the fit of assessment approach with selected system traits, to the identification and evaluation of relevant perturbations, to methodology for assessing general resilience, to the selection of factors contributing to resilience, and to the operationalization of transformative capacity.