Abstract

Adaptive Neuro-Fuzzy Inference System (ANFIS) blends advantages of both Artificial Neural Networks (ANNs) and Fuzzy Logic (FL) in a single framework. It provides accelerated learning capacity and adaptive interpretation capabilities to model complex patterns and apprehends nonlinear relationships. ANFIS has been applied and practiced in various domains and provided solutions to commonly recurring problems with improved time and space complexity. Standard ANFIS has certain limitations such as high computational expense, loss of interpretability in larger inputs, curse of dimensionality, and selection of appropriate membership functions. This paper summarizes that the standard ANFIS is unsuitable for complex human tasks that require precise handling of machines and systems. The state-of-the-art and practice research questions have been discussed, which primarily focus on the applicability of ANFIS in the diversifying field of engineering sciences. We conclude that the standard ANFIS architecture is vastly improved when amalgamated with metaheuristic techniques and further moderated with nature-inspired algorithms through calibration and tuning of parameters. It is significant in adapting and automating complex engineering tasks that currently depend on human discretion, prominent in the mechanical, electrical, and geological fields.

1. Introduction

The machine learning domain contains a wide variety of models based on the learning ability, adaptiveness, complexity, and scalability. Some of the popular techniques are Fuzzy Logic, Extreme Learning Machine, Boosting, Bagging, Artificial Neural Networks, etc. Many researchers used machine learning algorithms based on these techniques like regression, decision trees, random forest, stochastic gradient, Support Vector Regressors (SVR), etc. and its ensembles other optimization techniques [1]. Hybrids of such techniques have been proposed and developed that tend to solve their deficiencies as well as provide robustness and powerful prediction capabilities. One such technique with the inherent potential of both neural networks and fuzzy systems is ANFIS [2], which provides great estimation accuracy, i.e., low Mean Magnitude of Relative Error (MMRE) and high Prediction (PRED).

ANFIS is the most popular neuro-fuzzy model for approximating highly complex, nonlinear systems. The key aspects of ANFIS are the accuracy using the precise fuzzy modelling and interpretability, which improves its generalization ability. ANFIS has gained prominence amongst researchers for its robustness in modelling fuzzy sets into crisp inputs and providing crisp outputs from the fuzzy rules for reasoning purpose. Ironically, ANFIS has to balance the accuracy-interpretability trade-off [3]. The advantages and disadvantages of ANFIS have been discussed in Table 1. It is of considerable importance in ANFIS to find the type and number of membership functions, suitable to the process or system. ANFIS is generally very efficient until the number of inputs is below five [4]. Modern engineering systems have more inputs as the complexity of the problem increases, for instance, signal processing in a highly chaotic environment, flood susceptibility detection in watershed management, precise coordination of I&C systems in a nuclear plant, etc.

Table 1 
Advantages and disadvantages of ANFIS.

Originally, ANFIS was designed with the use of Gradient Descent (GD) and Least Square Estimation (LSE) for optimizing its parameters. GD is a very popular optimization algorithm that is commonly used to train neural networks. It uses backpropagation method to calculate gradients, thus having the easiest system of computation. LSE method of optimization is very common in regression-based models. It calculates the least sum of squared errors, finding the optimal coefficients of the errors. However, these are not efficient to model complex engineering tasks that are highly nonlinear in nature and require precise control over the systems. Then, it provides an opportunity to improve the capability of ANFIS.

GD is a basic optimization algorithm that suffers when the nonlinearity of the system increases. Hence, it may fail to find the global optima and remain trapped at local minima. For large datasets, redundant computations are performed for the same set of training data, slowing the convergence. It has high computational cost, when frequently updating the weights of the neural network, wasting computational resources.

LSE is a rudimentary optimization method that is very sensitive to outliers. Its performance is affected when the data is not normally distributed, which leads to overfitting in most cases. Also, LSE is more computationally expensive than GD, becoming slower with increasing complexity of the system.

ANFIS is plagued by issues inherent in its fundamental structure. The optimization algorithms thus used are instrumental in altering the performance of ANFIS. Metaheuristic techniques assist ANFIS in searching for solutions for optimal and accurate predictions. Metaheuristic techniques provide a high-level and problem-independent set of directives to develop optimization techniques. These techniques have been found superior to the traditional techniques of optimization. Their goal is to compute a “good enough” solution in a “small enough” computing time not subjected to combinatorial explosion [5]. Hence, the solution obtained is quick and efficient, enabling optimization of the problem definition at hand. The various research papers that have been reviewed have hybridized the standard ANFIS architecture to include such metaheuristic algorithms for optimizing ANFIS premise and consequent parameters [6]. These optimization techniques can improve the standard ANFIS architecture [7, 8].

This paper has been divided into 4 sections: Section 1 provides Introduction, Section 2 contains the review methodology, Section 3 contains the Results and Discussions, and Section 4 describes the Conclusion and Future research directions.

This table presents the advantages and disadvantages of the original ANFIS system, as designed by J.S. Roger Jang. These limitations are unsuitable for use in modern, real-world systems and, hence, need to be resolved to be deployed in production on the machines.

2. Method

In this section, we have discussed the classic ANFIS [2], various research questions, review inclusion and exclusion criteria, data sources description, and study selection process [9]. The steps are shown in Figure 1.


Figure 1 
Methodology flowchart of the review protocol. This paper adheres to the steps mentioned in the figure, formulating a sequential presentation of the Systematic Literature Review (SLR). This methodology follows the standard process of reviewing literature and their selection for assuring quality review to the peers.
2.1. Adaptive Neuro-Fuzzy Inference System

Adaptive Neuro-Fuzzy Inference Systems, developed in 1993 by J.S. Roger Jang, are widely regarded as a universal estimator or Takagi-Sugeno Fuzzy System. The Takagi-Sugeno Fuzzy model is a Type 3 Fuzzy Inference System, where the rule outputs are a linear combination of input variable along with a constant, and the final output is the weighted average of every rule’s output.

The IF-THEN rules for a 3-input Takagi-Sugeno system are described as follows.(i)Rule 1: IF x is A1, y is B1, z is C1, THEN (ii)Rule 2: IF x is A2, y is B2, z is C2, THEN (iii)Rule 3: IF x is A3, y is B3, z is C3, THEN  where x, y, z are the inputs in the crisp set; Ai, Bi, Ci are the linguistic labels; pi, qi, ri are the consequent parameters; f1, f2, f3 are the output fuzzy membership functions.

The standard ANFIS architecture, as given in Figure 2, consists of five layers of interconnected neurons, evident of artificial neural networks having alike functionalities. The architecture is briefly explained as follows.


Figure 2 
Standard structure of ANFIS. , , and are the weights of the neurons and , , are the normalized weights of the neurons.
2.1.1. Layer 1

It is the Fuzzification Layer where each neuron is an adaptive node and holds the fuzzy value of the crisp inputs.

The node output is calculated as follows:where μ is a membership function for the fuzzy sets Ai, Bi, Ci. Numerous membership functions exist, i.e., Gaussian, Trapezoidal, Triangular, etc. We prefer a bell-shaped function in ANFIS. Hence, the Gaussian function is the optimum choice. The formula for Gaussian function iswhere a, b, c are the premise parameters for the membership functions of ANFIS.

2.1.2. Layer 2

This is an Implication Layer where the neurons contain the product of inputs, i.e., the weight of premise parameters. The node output is calculated as follows:where is the weight of the neuron.

2.1.3. Layer 3

It is Normalizing Layer where the neurons are fixed and are normalized by the sum of weights of all neurons in this layer. The node output is calculated as follows:where is the normalized weight of the neuron.

2.1.4. Layer 4

This is the Defuzzification Layer where each neuron is also an adaptive node and holds the consequent parameters of the architecture. The node output is calculated as follows:

2.1.5. Layer 5

It is an Output Layer where a single neuron is present for output, which is the sum of all the inputs. The node output is calculated as follows:

Classical ANFIS favors hybrid learning process, where parameters are updated through two passes and use two different optimization algorithms.

During the forward pass, the consequent parameters are updated, when the inputs are provided to ANFIS, and the premise parameters are kept fixed, using LSE, the consequent parameters are updated in Layer 4, and the final output is calculated accordingly.

As the final output is calculated, the backward pass starts, during which the error is propagated back to Layer 1, and the premise parameters are updated. In this pass, the consequent parameters are kept fixed.

2.2. Research Questions

This review paper aims to summarize the current implication status of machine learning models. In this context, the following research questions (RQ) are proposed: RQ1: What are the various ANFIS hybrids? RQ2: What was the purpose of creating a hybrid ANFIS technique? RQ3: What areas of applications have utilized ANFIS hybrids in the real world? RQ4: What are the various optimization algorithm hybrids of ANFIS? RQ5: What are the possible future research directions in context to ANFIS current literature variants? RQ6: What are the current trends in research based on ANFIS techniques? RQ7: Which hybrid techniques are the most popular for ANFIS implementations?

2.3. Search Strategy

We employ two phases to search and download the studies.

2.3.1. Primary Search

Using the primary search phase, we adopt the following procedure:(i)Analyze leading terms within research questions(ii)Detect alternative terms and spelling for the terms(iii)Check the keywords in relevant books and journals(iv)Boolean AND to link leading terms and Boolean OR for alternative terms

2.3.2. Secondary Search

In the secondary search phase, references ignored during the primary search are reviewed. The search string used is, Hybrid AND (ANFIS OR “Neuro-Fuzzy” OR optimization) AND techniques AND (using OR with) AND (“Artificial Bee Colony” OR “Ant Colony Optimization” OR “Bat Algorithm” OR “Bees Algorithm” OR “Biogeography Based Optimization” OR “Cultural Algorithm” OR “Colliding Body Optimization” OR “Cuckoo Optimization Algorithm” OR “Crow Search Algorithm” OR “Cat Search Algorithm” OR “Differential Evolution” OR “Firefly Algorithm” OR “Genetic Algorithm” OR “Grey Wolf Optimizer” OR “Harmony Search” OR “Imperialist Competitive Algorithm” OR “Invasive Weed Optimization” OR “Moth Fly Optimization” OR “Mosquito Host Seeking” OR “Particle Swarm Optimization” OR “Simulated Annealing” OR “Satin Bowerbird Optimizer” OR “Subtractive Clustering” OR “Shuffled Frog-Leaping Algorithm” OR “Social Spider Optimization”).

The search strategy has been further refined using alternative terms and spellings in the search string. Boolean strings enable the discovery of all studies available in the databases, whereas the references present in the selected studies can be benefited from any missing studies. Such considerations, and suggestions, allowed us to employ a search string using Boolean operators OR and AND comprising possible alternatives for the terms. Besides, references of the primary studies served as sources for exploration of possible missing studies. The following sources of literature discovery were used for selecting primary studies:(i)PLOS One(ii)IEEE Xplore(iii)Springer(iv)ACM Digital Library(v)ScienceDirect(vi)Elsevier

These revered digital libraries are popular in the research community; hence, we found them suitable to include for compiling our data. The studies we focused on are between 1997 and 2019, with publications in the first quarter of 2020 also included. During the primary search phase, we examined and found 48 relevant studies. Afterward, in the secondary search phase, our discussed criteria allowed the identification of 79 additional relevant studies, missed during the initial searches. Thus, we selected 127 studies specific to our SLR, based on the conducted primary and secondary search phases. The title and abstract of these studies were considered for selection.

2.4. Study Selection

Every study identified based on titles and abstracts during the search strategy was moved through two phases for filtering the studies, such that the desired literature was obtained. The first phase contains inclusion and exclusion criteria for selecting studies of relevance and discarding others for our SLR. In the second phase, further filtering of the selected studies is based on the quality assessment criteria. The inclusion/exclusion criteria specific to our SLR are as follows:

2.4.1. Inclusion Criteria

The following are the inclusion criteria:(i)Studies that contain hybrid of ANFIS(ii)Studies that propose novel optimization techniques(iii)Studies that contain hybrid of optimization techniques

2.4.2. Exclusion Criteria

The following are the exclusion criteria:(i)Studies with little relevance in case of similar content(ii)Studies containing only Deep Learning techniques or Fuzzy Logic techniques(iii)Studies published as a thesis

2.5. Quality Assessment Criteria

Every study was examined against the quality assessment criteria to validate their plausibility and significance. In particular, after the application of the inclusion/exclusion criteria, we have carefully observed the candidate studies relevant to our purpose. Studies having low quality have been vetoed. We have used the following questions to define the criteria for quality assessment. Q1. Are the established aims in the research thoroughly defined? Q2. Has the study been cited before? Q3. Is the experiment applied against benchmark functions or any application areas? Q4. Are the algorithms validated against standard accuracy measures? Q5. Are optimization techniques provided with comparative analysis? Q6. Are limitations catering to the study analyzed explicitly? Q7. How recently the study has been published? Q8. Are the journals indexed in SCI or Scopus?

For assessing the quality of the studies based on these criteria, we provide only three answers with the scores given in (7).

The quality score for a given study is calculated by taking the mean of the scores against the questions answered and considered for selection as per the threshold in

The quality assessment score for the studies is not based on fuzzy linguistic values because of the relative ease provided by the crisp set in our criteria. As we assess the quality based on the mean of the scores in the range defined in (8), it allows a comfortable examination to us. After several rounds of discussion with regards the quality assessment criteria, all the authors acknowledged the proposed system of scoring. To denote the simplicity of the system, we take an example. Suppose a study receives the subsequent quality assessment scores for the eight questions: Q1 (1), Q2 (0.5), Q3 (0.5), Q4 (0.5), Q5 (1), Q6 (1), Q7 (1), and Q8 (1)

The total score is 6.5, and their mean is 0.8. This validates the study for selection as it is above the acceptable threshold defined by our system.

In another example, if the following scores are obtained for a particular study: Q1 (0.5), Q2 (0.5), Q3 (1), Q4 (0), Q5 (0), Q6 (0.5), Q7 (0.5), and Q8 (1)

The total score is 4 and their mean is 0.5; hence, this study cannot be included and is filtered. In general, with a minimum total score of 5, the study is suitable for selection in the SLR. The application of these quality assessment criteria excluded 11 studies.

2.6. Data Extraction

One issue we recognized when searching the studies for SLR was the use of partial terms. For example, some of the studies contained the term particle swarm and genetic, but not optimization or algorithm. Another issue we recognized was the inclusion of terms in the abstract, but not in the title. As we examined further, we found some studies including the terms for comparative purpose, as opposed to being the focus of the studies. It is worth noting that our research questions are not necessarily answered by all the selected studies. Such note of importance prompted us to analyze and assign a score to every study. Based on the number of RQs addressed, each study received a score accordingly, which formed the basis for the final quality assessment score. The score for every study ranges from 0 to 1, where 0 is the minimum score, while 1 is the maximum score a study can receive. The higher value of score increases the sincerity of the study. Each study scores one point for each research question addressed. Since we have seven research questions, a study can achieve seven points at maximum. Similarly, the quality assessment score of every study is based on (7) with mean according to the threshold in (8) for the final score. Hence, for addressing the research questions, a study can achieve a maximum score of 7 and a quality assessment score as 1.

2.7. Data Synthesis

For the classification and arrangement of every piece of information from the selected studies related to our research questions, we employ data synthesis. We have primarily adopted two methods to synthesize our results:(i)Narrative: we tabulate the results, after analyzing the data, incorporating various charts. RQ2, RQ3, RQ4, RQ5, and RQ6 belong to this category.(ii)Vote counting: we make some comparisons between the various models that are having higher research potential. RQ1 and RQ7 belong to this category.

2.8. Threat to Validity
2.8.1. Study Selection Bias

Our defined search string is used in selecting the relevant studies for our SLR. We gave the best of our efforts in phrasing the search string in correspondence to our research questions. Yet, an existing possibility, where some relevant studies may have been missed, as per the fact that some studies had different keywords, in their title, abstract, and keywords, should be taken into account. Though we have emphasized avoiding such possibilities, using references from the bibliography of particular studies for the selection of all relevant studies, yet, there exists a probability that an important study may have been missed and as such is considered a threat.

2.8.2. Subjective Quality Assessment

Quality assessment defines the criteria, by which we include and exclude the studies. This SLR also includes a possibility, in which several good quality studies might have been excluded. We try to minimize such threats, by scoring the studies based on our discussed criteria and making a final assessment based on the mean of those scores, thereby excluding only those studies, which fall below our set threshold criteria.

3. Results and Discussions

The proposed research questions are discussed in this section.

3.1. What Are the Various ANFIS Hybrids? (RQ1)

The optimization algorithms are hybridized with existing techniques as a common practice. These techniques can be applied to various emerging domains. ANFIS can also be hybridized with such techniques proving its viability as a universal estimator. Table 2 shows the summarized view of various ANFIS hybrids.

Table 2 
Summary of prevailing ANFIS hybrids.

This table summarizes the various ANFIS hybrids, using optimization techniques, and their real-world applications. ANFIS widely adapts backpropagation for parameter optimization. Development of optimization algorithms revitalized the efforts to increase the accuracy of these systems, implicating that they can also address unique real-world problems, with human-like processing capability.

3.2. What Was the Purpose of Creating a Hybrid ANFIS Technique? (RQ2)

Real-world data is multidimensional, complex, and huge. The standard ANFIS uses Least Square Estimation and Gradient Descent to optimize its parameters, which can cause inaccurate prediction. This is due to the limitation that these algorithms converge slowly and compute time-consuming mathematical operations. To provide optimized solutions quickly in a large dataset, hybrid ANFIS techniques have been developed.

3.3. What Areas of Applications Have Been Touched by ANFIS Hybrids in the Real World? (RQ3)

ANFIS and its hybrids find widespread applications in several key areas of sciences and engineering. Figure 3 demonstrates the share of applicability of ANFIS hybrids. From all the domains, Electronics has mostly embodied ANFIS followed by Geological Engineering. This suggests that ANFIS can be used to model complex real-world problems, e.g., software effort estimation [34] and intelligent systems.


Figure 3 
Share of application areas using ANFIS hybrids. This figure represents the average share of the various fields, where ANFIS has been implemented in real-world problems. ANFIS presents a wider potential for large scale, consumer-level product deployment, assisted by the advent in the field of Internet of Things (IoT) and 5th Generation (5G) Networks.
3.4. What Are the Various Optimization Algorithm Hybrids of ANFIS? (RQ4)

Table 3 summarizes variants of the ANFIS hybrid techniques. The optimization techniques that extended the capabilities of ANFIS have been enhanced by hybridizing them with other optimization algorithms or various techniques to further improve their learning capability. ANFIS-PSO [28] has the most potential of advancing its capabilities because PSO [55] can be easily modified to include hybridized extensions. Hence, PSO can be combined with ANFIS when creating a variant, for accelerated computing ability. FFA [56] has also been favored by the researchers for extending capabilities of ANFIS.

Table 3 
Optimization algorithm hybrids with their ANFIS counterparts.
3.5. What Are the Possible Future Research Directions in context to ANFIS Current Literature Variants? (RQ5)

There are several hybrids of optimization techniques that are not yet implemented with ANFIS and are mentioned in Table 4. These algorithms are mostly hybridized variations of the standard optimization algorithms. PSO and BAT are widely hybridized and used to extend standard ANFIS architecture. ANFIS is a complementing framework for all technological paradigms and cross-cutting concerns. The exponential growth of data in recent years creates a thrust area to address issues of faster data processing capabilities, which can be handled well by ANFIS.

Table 4 
Proposed hybrids of ANFIS implemented models.
3.6. What Are the Current Trends in Research Based on ANFIS Techniques? (RQ6)

Figure 4 provides a trend of publications related to ANFIS techniques and its hybrids. The research remained stagnated for 14 years, up until 2007, when it was first combined with a hybrid of another popular technique, PSO, called Adaptive Weighted PSO. This suggests the viability of ANFIS in solving complex real-world problems as we move into automation and developing intelligent systems. The reason for the delayed usage of ANFIS in the scientific community is the presence of fewer data in the initial years of its invention. The research publication graph observes exponential growth as it delivers promising, effective, and accelerated results, a.k.a. solutions for various engineering and science domain optimization issues.


Figure 4 
History of publications on ANFIS (1993–2020). The wide research potential of the optimization algorithms led to an increase in publications in ANFIS.
3.7. Which Hybrid Techniques Are the Most Popular in the Case of Implementations? (RQ7)

Figure 5 shows the count of ANFIS hybrids applied in different application areas. ANFIS-PSO has gained the highest popularity in the research community. The popularity of PSO has been exemplified here followed by ANFIS-ABC, ANFIS-FFA, and ANFIS-SC. The other hybrids as shown in Figure 5 are also successful trials.


Figure 5 
Common implemented ANFIS hybrids. These selective hybrids are immensely implemented, in an attempt to develop solutions for several real-world problems.

4. Conclusion and Future Research Directions

ANFIS is one of the most promising algorithms to model human knowledge effectively, asserting its capabilities in complex problems that require manual intervention from humans. We have provided an exhaustive list of the most prominent ANFIS hybrids till date and their intended purpose by the authors. We further discussed the need to hybridize ANFIS, its usage in the current scenario and future scopes in modern engineering sciences. With the advent of metaheuristic optimization techniques, we discussed its advantages as compared to classical optimization methods and provided a comprehensive list of hybrids of these algorithms, which had been used further in ANFIS. The popularity of such metaheuristic techniques leads us to provide an analysis of the most sought-after algorithms that were used to hybridize ANFIS, with ANFIS-PSO and ANFIS-ABC as the topmost choices.

We made an attempt to provide significant insights into the fields of implementations for ANFIS. It is assured that the future scope of research in ANFIS is having high potential. Electronics, Communication, and Geological fields of engineering have strived to adopt ANFIS, based on its amazing capabilities as a universal estimator. ANFIS can be opted for implementing Artificial General Intelligence; hence, its popularity is predicted to increase, as analyzed earlier in this paper.

Data Availability

The data shall be made available upon request.

Conflicts of Interest

The authors declare that they have no conflicts of interest.