Improve performance and robustness of knowledge-based FUZZY LOGIC habitat models

https://doi.org/10.1016/j.envsoft.2021.105138Get rights and content

Highlights

  • We identify fuzzy logic limitations and sources of error and bias on the results.

  • We propose a framework to improve fuzzy model performance and reduce uncertainty.

  • Clear model definitions and objectives can reduce bias due to expert misinterpretation.

  • Well-defined rules and the use of the OR operator can reduce model complexity.

  • Model performance can be improved by dealing with sources of bias.

Abstract

Previous criticisms of knowledge-based fuzzy logic modelling have identified some of its limitations and revealed weaknesses regarding the development of fuzzy sets, the integration of expert knowledge, and the outcomes of different defuzzification processes.

We show here how expert disagreement and fuzzy logic mechanisms associated with the rule development and combinations can positively or adversely affect model performance and the interpretation of results. We highlight how expert disagreement can induce uncertainty into model outputs when defining fuzzy sets and selecting a defuzzification method. We present a framework to account for sources of error and bias and improve the performance and robustness of knowledge-based fuzzy logic models. We recommend to 1) provide clear/unambiguous instructions on model development, processes and objectives, including the definition of input variables and fuzzy sets, 2) incorporate the disagreement among experts into the analysis, 3) increase the use of short rules and the OR operator to reduce complexity, and 4) improve model performance and robustness by using narrow fuzzy sets for extreme values of input variables to expand the universe of discourse adequately. Our framework is focused on fuzzy logic models but can be applied to all knowledge-based models that require expert judgment, including expert systems, decision trees and (fuzzy) Bayesian inference systems.

Introduction

Fuzzy logic is a knowledge-based modelling method that uses linguistic variables obtained from human operators and/or expert knowledge as model inputs with gradual (fuzzy) boundaries rather than discrete (crisp) numeric ones (Mamdani, 1977; Mamdani and Assilian, 1975). It mimicks an analytical decision-making process similar to human thinking (Shrestha and Simonovic, 2010; Zadeh, 1965). This method handles data imprecision and provides the ability to deal naturally with the vagueness of information (qualitative and linguistic data input) through an easily interpretable rule-based system using inference methods. The fuzzy inference method allows to reach an integrative conclusion deduced from a collection of imprecise premises by using IF-THEN rules (e.g. Pal and Mandal, 1991). Mamdani and Takagi-Sugeno inferences systems are the most commonly used processes for treating a set of given inputs to an output. Fuzzy logic thus provides the capacity to efficiently deal with information from data compiled in a qualitative rather than quantitative way, incomplete databases, or imprecise knowledge (Ahmadi-Nedushan et al., 2008). This ability is based on the integration of gradual changes in environmental variables, allowing simultaneous membership of a given numerical (crisp) value to more than one linguistic (fuzzy) variable category at various degrees (Van Broekhoven et al., 2006). This advantage and the easy incorporation of expert knowledge using the Mamdani inference system is one reason for the increasing popularity of this method in environmental sciences and ecosystem management (Table SI1). Indeed, ecological processes 1) often follow continuous monotonic trends rather than discontinuous dichotomous jumps (Jorde et al., 2001; Kerle et al., 2002), 2) can spread over large scales or periods, or 3) include varying, often unknown and unpredicted sources of uncertainty like stochastic variation, instrumental imprecision or subjectivity (Salski, 1992), making fuzzy logic approaches particularly suitable to describe these processes.

When validated by field observations, expert-knowledge fuzzy models regularly provide high predictive power (Adriaenssens et al., 2006; Bock and Salski, 1998; McLaughlin et al., 2006). However, some models have significant uncertainties as highlighted by low correctly classified instances <50% (Muñoz-Mas et al., 2016), high observed abundance associated with low predicted HSI (Fukuda et al., 2011; Theodoropoulos et al., 2018a), low coefficient of determination (R2), small proportion of explained abundance (Beaupré et al., 2020) or substantial difference between predictions and observed values (Mouton et al., 2008). Complex animal behaviour, inter- and intra-annual environmental variability that strongly influences ecological processes and populations, and stochastic events, may partially explain model weaknesses. Weak model validation can also result from knowledge gaps of the habitat preferences of specific taxa and the use of invalid validation protocols or ill-developed models. Expert knowledge strengths and weaknesses have been mainly reviewed by Drescher et al. (2013). Surprisingly, beyond the recurrent misunderstandings and misconceptions about this method (Zadeh, 2008), few criticisms exist about the application of the method itself (e.g. Mendoza and Martins, 2006). Therefore, important pitfalls remain regarding fuzzy logic application for HSI and are yet to be addressed (Table 1).

Using non-discrete classification offers a clear advantage when studying ecological processes that rarely follow a dichotomic trend (Singh et al., 2013; Zadeh, 1965, 2008). However, such methods need to be analyzed to improve their robustness and decrease the uncertainties associated with model development. As the application of fuzzy logic quickly spreads in animal habitat modelling (Supplementary Table S1), we use fuzzy logic for fish habitat modelling as a case study to improve modelling relying on expert-based knowledge and provide examples for the discussion points. We provide a critical overview of the methodological pitfalls that may affect the performance of fuzzy models using Mamdani inference system but applicable to any knowledge-based methods. We then propose solutions to each issue to reduce negative influences on model outcomes and increase the model accuracy. More specifically, based on a brief review of fuzzy logic method developments, we discuss inherent challenges related to 1) the definition of model variables/inputs to be used by experts, 2) the consideration of expert disagreement and potential misunderstandings, 3) the sources of inherent uncertainties of fuzzy logic mechanisms, such as input variables, fuzzy sets and rules definitions and 4) the handling of extremes conditions and model complexity. We finally propose solutions to improve the accuracy and robustness of methods such as fuzzy logic or Bayesian inference that rely on expert knowledge.

Section snippets

Fuzzy logic rudiments

To better understand the sources of uncertainties inherent to the method itself, we briefly remind here how data are fuzzy-transformed and processed to return a numeric model output.

The first fuzzy logic step consists of transforming numerical data into fuzzy sets, a process called fuzzification. This step requires a precise definition of each set based on expert knowledge to determine membership functions and boundaries (Fig. 1) for each set of every variable. Membership functions are commonly

A critical analysis of the fuzzy model process

Following the expert-based fuzzy logic model development steps, we develop a decision framework to demonstrate and discuss potential pitfalls and underlying sources of uncertainties linked with the fuzzy inference process (Fig. 2). From this framework, we discuss the importance of expert knowledge selection and validation, the implications of excessively broad variables and sets, and potential benefits of comparing different defuzzification methods, combining critical aspects that have not been

Conclusion

Although we discuss a method suitable for linear ecological changes, we want to acknowledge that not all ecological patterns follow such trends. Moreover, understanding the discontinuities can provide more critical knowledge of ecological phenomena as it is also essential to understand non-linear dynamics, feedback loops and multiple stable equilibria so we can better what trigger changes in a system (e.g. decrease in biodiversity) and how to mitigate these changes (e.g. restoration effort to

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

The authors are thankful to Rafael Muñoz-Mas for providing feedback on the manuscript. They also express their most heartfelt thanks to Andre St-Hilaire, who has been a tremendous mentor and for his support of Dr. Mocq's and Dr. Ouellet's academic careers.

References (53)

  • A.M. Mouton et al.

    Knowledge-based versus data-driven fuzzy habitat suitability models for river management

    Environ. Model. Software

    (2009)
  • A.M. Mouton et al.

    Optimization of a fuzzy physical habitat model for spawning European grayling (Thymallus thymallus L.) in the Aare river (Thun, Switzerland)

    Ecol. Model.

    (2008)
  • R. Muñoz-Mas et al.

    Generalized additive and fuzzy models in environmental flow assessment: a comparison employing the West Balkan trout (Salmo farioides; Karaman, 1938)

    Ecol. Eng.

    (2016)
  • M.M. Rajabi et al.

    Efficient fuzzy Bayesian inference algorithms for incorporating expert knowledge in parameter estimation

    J. Hydrol

    (2016)
  • M.S. Reed

    Stakeholder participation for environmental management: a literature review

    Biol. Conserv.

    (2008)
  • A. Salski

    Fuzzy knowledge-based models in ecological research

    Ecol. Model.

    (1992)
  • E. Van Broekhoven et al.

    Fuzzy rule-based macroinvertebrate habitat suitability models for running waters

    Ecol. Model.

    (2006)
  • V.C. Yen

    Rule selections in fuzzy expert systems

    Expert Syst. Appl.

    (1999)
  • L.A. Zadeh

    Is there a need for fuzzy logic?

    Inf. Sci.

    (2008)
  • L.A. Zadeh

    Fuzzy sets

    Inf. Control

    (1965)
  • S. El Adlouni et al.

    Regularized Bayesian quantile regression

    Commun. Stat. Simulat. Comput.

    (2018)
  • B. Ahmadi-Nedushan et al.

    Instream flow determination using a multiple input fuzzy-based rule system: a case study

    River Res. Appl.

    (2008)
  • W. Aspinall

    A route to more tractable expert advice

    Nature

    (2010)
  • Y. Bai et al.

    Fundamentals of fuzzy logic control — fuzzy sets, fuzzy rules and defuzzifications

    Adv. Ind. Control

    (2006)
  • M.A. Burgman et al.

    Expert status and performance

    PloS One

    (2011)
  • M. Drescher et al.

    Toward rigorous use of expert knowledge in ecological research

    Ecosphere

    (2013)
  • Cited by (0)

    1

    Both authors equally contributed to the manuscript.

    View full text