Exploring the Key Factors for Preventing Public Health Crises Under Incomplete Information

Abstract

The spread of COVID-19 has triggered one of the largest pandemics in modern human history. Humanity is still in the incomplete information period for this infectious disease, and how to effectively deal with such a major public crisis is a crucial problem. Although there are divergences in human natural semantics, the incomplete information increases it. Therefore, this study integrates the neutrosophic set and the Decision-Making Trial and Evaluation Laboratory (DEMATEL) methods to explore the key factors which would prevent expansion of the epidemic in the face of incomplete knowledge. The neutrosophic set technique is an effective tool for the representation of the ambiguity of natural human semantic expression, for the analysis of incomplete, uncertain, and inconsistent information. DEMATEL is used to explore the causes and effects between factors and to generate an influential network relationship map. The results of analysis can help the government and relevant organizations to understand the cause and effect relationship between the factors and set appropriate prevention strategies. The results of this study show that the incorporation of neutrosophic set theory leads to a more meaningful evaluation under incomplete information. “Detect” is a key factor affecting the entire system. The results of this study contribute to the advancement and development of scientifically based decision-making by helping governments and relevant organizations to understand the causal relationships between factors, to set appropriate prevention strategies.

1 Introduction

At the end of 2019, COVID-19 began to spread rapidly around the world and has since reached pandemic proportions, fortunately a rare event in modern human history. Experts from all over the world hold many different views about the cause and future development of this infectious disease [1,2,3,4]. It is worth noting that the public health systems of all countries in the world are fully committed to using national resources to fight the pandemic. At this moment though, when human medical technology has not been able to successfully defeat this disease, we can still only rely on traditional emergency response measures to resist the virus. Experts from all over the world have frequently discussed the past response to public health emergencies of international concern, PHEIC [5,6,7].

The United States Department of Homeland Security drafted the National Response Framework (NRF) in 2008. This framework proposes some guiding principles to facilitate the communication and coordination of units at all levels (regional, state and federal). It clarifies the key principles of the state's response, describes the roles and responsibilities of each unit, and determines the key structures [8, 9]. Nelson et al. (2007) conducted a holistic assessment of public health emergency preparedness and discussed many of the concepts, tools, and challenges involved. The results show that most of these tools have some drawbacks, many relying on written self-assessment, mainly focused on the structure of emergency preparedness. Although there have been many preparations made and studies of public health emergencies internationally, few have discussed the complexity and interaction between factors in the existing evaluation framework [10].

As early as 2005, the World Health Organization (WHO), in their International Health Regulations (IHR), drafted an evaluation system the so-called joint external evaluation (JEE) assessment framework. This assessment framework conducts an overall assessment of 19 technical fields and multiple departments within a country. It is an objective, voluntarily, and independent assessment. The JEE scores the performance of various indicators on a 5-point scale. The evaluation criteria span multiple technical fields and often require cross-departmental cooperation [10, 11]. The JEE assessment framework is more comprehensive than traditional public health assessment frameworks, and has demonstrated a certain degree of representativeness when used for assessments in many regions/countries around the world. Therefore, this study will use the JEE as the evaluation framework to conduct an overall discussion of the national public health emergency response system. It should be noted that in the face of an unknown public health emergency, in this case COVID-19, it is crucial to know how to respond, prevent and reduce losses. Therefore, being able to quickly identify the key elements to prevent the spread of the epidemic given limited resources and incomplete information is essential.

In recent years, there have been many different techniques and approaches to explore key elements, such as: AHP, ANP, etc. However, each technique has its advantages and limitations. It’s worth noticed that the Decision-Making Trial and Evaluation Laboratory (DEMATEL) method is one of the tools commonly used to explore key factors and has been widely applied in different research fields in recent years [12]. Pairwise comparisons are used to determine the influence relationship between factors from which to estimate the influence relationship for the entire evaluation system. The interactive influence of all factors is fully considered in the calculation process. The results are visualized graphically to clearly show the degree of influence between factors and the causality of the factors [12,13,14,15,16,17,18]. However, the decision-making evaluation process in this method relies upon on experts’ judgements for analysis and evaluation. Since these experts are humans, there is a problem with ambiguity in natural semantic expression although some efforts have been made to alleviate the problem of ambiguity. Many studies have used fuzzy theory to remedy this type of problem [19,20,21,22,23,24]. However, the traditional fuzzy theory cannot completely overcome the problem of incomplete information in the decision-making process. Human thinking has the characteristic of uncertainty. Not only are there ambiguities in natural semantic expression, but also many uncertainties and complexities in the decision-making problems [25]. To this end, many useful tools have been developed to model vague and uncertain information, including: fuzzy sets, FS [19, 20], intuitionistic fuzzy sets, IFS [20], interval type-2 FN [26], IFSs [27,28,29], hesitant fuzzy linguistic term sets [30, 31] etc.

The neutrosophic set is an extension of these powerful techniques, which considers a truth-membership, an indeterminacy-membership and a falsity-membership at the same time [25, 32, 33]. In addition to effectively resolving the ambiguity of natural semantic expression, it can also deal with incomplete, uncertain and inconsistent information. How to effectively prevent the spread of COVID-19 during the pandemic is a complex, multi-attribute decision-making problem. The problem is very complicated and contains many unknowns and uncertainties. Therefore, for the analysis, we combine the neutrosophic set with the DEMATEL method to improve the efficiency of the evaluation process and reduce the interference of inconsistent information [32]. Few studies on public health emergencies have used the neutrosophic DEMATEL method to assess the causality of preventive elements.

The results of this study can help to improve human emergency response capabilities and are applicable to planning for previously unknown public health emergencies. The evaluation results will facilitate real world decision-making. The contributions include the following:

• (1) The integration of the neutrosophic set and DEMATEL techniques will be more appropriate under incomplete information, and the results will be more suitable for decision-making problems in the real world.

• (2) The framework proposed by the WHO is adopted to help facilitate international discussion and communications.

• (3) To verify the applicability of this method, a case study is conducted using data from Taiwan, a country with excellent performance in preventing the spread of COVID-19.

The rest of this article is organized as follows: Sect. 2 introduces the existing literature. The procedure and calculation steps are explained in Sect. 3. Section 4 describes the problem and case. Section 5 presents the results, managerial implications and a discussion. The sixth section summarizes the study and put forward future topics.

2 Literature Review

2.1 Evaluation of Public Health Emergency Response System

There are many national public health emergency response system evaluation frameworks such as the Global Health Security (GHS) index, eSPAR, INFORM score, International Health Regulations (IHR) index, Joint External Evaluation (JEE), Epidemic Preparedness Index (EPI), Ready Score, and so on [34]. Among them, the JEE, a new WHO tool, is designed to help countries evaluate their ability to prevent, detect and respond to infectious disease outbreaks in accordance with the IHR [35,36,37]. Prior to 2015, each country had to self-report to the World Health Assembly on its implementation of the IHR [37, 38]. However, several review committees and various expert groups of the IHR suggested that in addition to the mandatory annual report, three voluntary components should be included, one of them being the JEE [38, 39], developed in 2015, based on Global Health Security Agenda assessment tool [38, 40] and the Organization for Animal Health (OIE) Performance of Veterinary Services [38, 41].

The JEE tool was developed by a group of public health experts from all over the world. The first version of this tool was released in February 2016. A revised version was formally released in mid-2017 after discussions in multiple meetings of experts in various technical fields around the world [39, 42]. The main changes in the second edition of JEE included the addition of two financing indicators, the merging of two indicators in the legislation into one, and the renaming of three technical fields. Compared with the first version, the second version also contained more explanations and annotations of the indicators, attributes, questionnaires, etc. In addition, it also distinguished between animals and humans, and between the scoring methods for humans and animals. Finally, the second version modified the introduction of indicators, such as those linking public health and security authorities, risk communication, points of entry, chemical events and radiation emergencies [38, 42].

2.2 Joint External Evaluation Tool

The JEE Tool divides indicators into four dimensions: prevention, detection, response, and IHR-related hazards and points of entry. There are 19 technical fields covered inside the tool, which represent the public health functions related to health and safety [39, 43]. Since the International Health Regulations came into effect in 2007, countries have made tremendous efforts to strengthen their ability to prevent, detect and respond to public health emergencies. Prevention, detection, and response are essential for controlling public health risks, including outbreaks of infectious diseases [34, 42]. The dimension for “prevent” in the second edition tool includes “national legislation, policy and financing, IHR coordination, communication and advocacy, antimicrobial resistance, zoonotic disease, food safety, biosafety and biosecurity and immunization.” The dimension of “detect” includes “national laboratory system, surveillance, and reporting and human resources.” The dimension “response” includes “emergency preparedness, linking public health and security authorities, medical countermeasures and personnel deployment, and risk communication.” The dimension “IHR related hazards and points of entry” contains “points of entry, chemical events and radiation emergencies”; see Fig. 1.

Fig. 1

Joint external evaluation tool second edition

The final goal of the dimension “prevention” is to ensure that sufficient funds are provided for the implementation of the International Health Regulations (IHR) through means of inclusion in the national budget or other mechanisms. This is mainly designed to measure whether the current legislation (laws, regulations, administrative requirements, policies or other government instruments) of the region/country is sufficient to support the implementation of IHR and whether the region/country has sufficient funds to respond to all public health emergencies. It contains seven guidelines: C₁ (National legislation, policy and financing), C₂ (IHR coordination, communication and advocacy), C₃ (Antimicrobial resistance), C₄ (Zoonotic disease), C₅ (Food safety), C₆ (Biosafety and biosecurity), C₇ (Immunization).

The final goal of the dimension “detection” is to monitor the national laboratory system including all relevant departments, especially, for human and animal health, as well as effective modern medical spots and laboratory-based diagnostics. The main purpose of this dimension is to measure whether the region/country has a nationwide laboratory system and weather the quality of the system meets the requirements. It contains four criteria: C₈ (National laboratory system), C₉ (Surveillance), C₁₀ (Reporting), C₁₁ (Human resources).

The final goal of the dimension “response” is to ensure the response capability of the country/region's “emergency preparedness”, which refers to the operational capabilities in the face of any emergency hazards (i.e., biological, chemical, radiological and nuclear, natural, other technological and societal hazards). It is mainly to measure whether the region/country has multi-hazard emergency preparedness capabilities. Evidence can be obtained from emergency response plans, risk profiles, resource mapping and exercises, post-event investigations, and other assessments. It contains five criteria: C₁₂ (Emergency preparedness), C₁₃ (Emergency response operations), C₁₄ (Linking public health and security authorities), C₁₅ (Medical countermeasures and personnel deployment), C₁₆ (Risk communication).

The final goal of the dimension “IHR related hazards and points of entry” is to ensure that national borders (airports and ports) and core capacities (designate and maintain core capacities at international airports and ports) are managed appropriately. It is mainly designed to measure whether the region/country has core management and control capabilities for border management and public health emergency response plans. It contains three criteria: C₁₇ (Points of entry), C₁₈ (Chemical events), C₁₉ (Radiation emergencies).

Due to space limitations, please refer to the Joint External Evaluation tool (JEE tool)-second edition prepared by the WHO in 2018 for the purpose and measurement of each criterion.

2.3 Decision Modeling of a Public Health Emergency Response System

The rapid global spread of COVID-19 has challenged existing human science and technology to find new treatments for COVID-19 and for the prevention of transmission.

Most of the research about preventing COVID-19 has focused on clinical treatment, drug regimens, vaccine development, and preventive measures. Reference [44] conducted a review of existing treatment and prevention programs, and found 335 relevant studies, 309 trials evaluating therapeutic molecules, devices and other management. In addition, we found 23 studies evaluating drug and vaccine development and other prevention strategies, while 3 studies examined both topics. A review of studies about the prevention of COVID-19 showed that there are several possible main treatment interventions, including protease inhibitors [45, 46], RNA polymerase inhibitors [47, 48], anti-malaria drugs [49], immunomodulators and anti-inflammatory drugs [50, 51], membrane fusion inhibitors and inhibitors of ACE2 receptor connections [52], and cell therapies [53]. In terms of preventative measures, individual quarantine and active monitoring are deemed core strategies [54]. Kosfeld et al. [55] used a spatial difference-in-difference approach to analyze six types of containment measures: school closure, establishment closure, shopping mall and non-essential retail store closure, restaurant closure, contact restrictions, and facemask duty. Research has shown that contact restrictions and school closures will help to flatten the infection curve. Some have used mathematical modelling to compare and analyze public health measures. The results show that quarantine is very important to reducing the morbidity and mortality rates during the COVID-19 pandemic [56]. According to Nussbaumer-Streit et al. [57], quarantine should be implemented as soon as possible, and integrated with multiple public health measures. In the absence of reliable vaccines and treatments, the national public health emergency response system is considered the most important resource, for all countries in the world [5,6,7]. Experts in various fields have proposed a variety of different decision-making techniques designed to prevent or slow down the spread of the epidemic.

Therefore, experts in various fields have proposed many different technologies for decision-making to prevent the spread of the epidemic.

Based on an analysis of the literature, Davies and Wenham [58] advocated that any effective strategy for preventing the epidemic should use “international relations” as a point of departure to discuss the response during a national public health emergency. Their results show that international relations are one of the keys to influencing public health decision-making and technical policy coordination. In another study, researchers evaluated the emergency response ability of the national public health system and confirmed some key indicators. The results show that the experience of countries such as Vietnam, South Korea and Germany are countries that successfully controlled the epidemic is worth learning from in other countries around the world [59]. Boyd et al. [60] used a linear regression model to confirm the correlation between GHSI indicators and the overall global spread of an infectious disease. Gupta et al. [35] used principal component analysis to create a standardized JEE index score, which represents the cumulative performance of each indicator. Then, the index score identifies the potential drivers affecting performance. Ghasemi et al. [61] argued that many studies in the past only compared indicators in a single country and only considered the statistics of virus transmission and mortality. The lack of consideration of the impact of other variables may lead to misleading decisions. Therefore, they used dynamic data envelopment analysis to determine the key success factors that would affect national emergency response capability.

From a review of the above literature, we find out that in the decision-making problem of preventing COVID-19 from spreading, it is crucial to explore the key factors of influence. Various different techniques to explore the essential factors and control the spread of COVID-19, however, most of these techniques have some limitations, such as unreasonable assumptions or independence limitations. DEMATEL has been a useful tool to discover the essential factors and discuss the network relationship between factors. This method explores the impact of the entire system based on the impact relationship between indicators, and finally presents the causal relationships within the entire system in a visual way [12,13,14, 62]. Therefore, in this study we will use the DEMATEL method to discuss the factors that will prevent the spread of the epidemic.

2.4 Correction of Incomplete Information

Although the DEMATEL method has contributed to the exploration of key factors, it still has certain limitations. For example, the decision-making process requires expert judgements as the input data. However, faced with an unknown situation like the COVID-19 epidemic, even experts have difficulty formulating opinions using a crisp number. The fuzzy set (FS) theory was proposed by Zadeh [63, 64] as a way to overcome the ambiguity and uncertainty encountered using natural semantics in the decision-making process and its ability to describe situations where the data are uncertain or imprecise has certainly been a major contribution to the development of decision science.

However, there is still a lot of room for improvement when using the traditional fuzzy theory in practical applications [33]. Therefore, in recent years, a series of extensions to the fuzzy theory have been made including to the following extensions: interval value fuzzy sets (IVFS), hesitant fuzzy sets (HFS), intuitionistic fuzzy sets (IFS), interval value intuitionistic fuzzy sets (IVIFS), intuitionistic type-2 fuzzy sets (IFS2), picture fuzzy sets (PFS), spherical fuzzy sets (SFS), etc. These extensions can be roughly divided into several types and discussed as: membership and non-membership functions; single and interval values; and positive, negative and neutral elements. There is no denying that these extensions have contributed to the practical applications of fuzzy set theory.

However, classical ambiguity remains at the level of truth-membership; ambiguity and inconsistent information are not taken into consideration [33, 65]. The neutrosophic set (NS) theory was proposed to improve the original fuzzy theory [66]. In the NS theory, three independent membership functions (i.e., false, true and uncertainty) need to be considered [67], which can handle uncertain and inconsistent information [68]. However, the quantitative process makes the decision-making process very complicated, which is why, in this study, single-valued neutrosophic sets (SVNS) are used, that not only can maintain the original effect, but also reduce the complicatedness of the decision-making process [32, 33, 69, 70].

Based on a comprehensive review of the literature, we can infer that the JEE tool is the most appropriate framework for evaluating a national public health emergency response system. Although there have been many studies aimed at the prevention of COVID-19, most of these have focused on performance evaluation and the finding impact/driving factors. Our study considers uncertain COVID-19 information and combines the SVNSs with the DEMATEL method to explore the complex relationship between factors. The derived implications can help governmental authorities set strategies to prevent the spread of COVID-19.

3 Methodology

Although a lot of research about how to control the spread of COVID-19 has been conducted worldwide, there is still room for improvement. There is a gap in the development of human science and technology which will cause uncertainty in the decision-making process for public health management. How to evaluate uncertainty in the decision-making process has a significant impact on the results of decision-making. Here, the single-valued neutrosophic set and DEMATEL technologies are combined to fully consider incomplete, uncertain and inconsistent information in the decision-making process. The characteristics and calculation steps of this method are explained in this section. The method is divided into four stages, as shown in Fig. 2. The objectives of the four stages are:

• (1) Develop goals and define the evaluation framework,

• (2) Evaluation of incomplete, uncertain and inconsistent information,

• (3) Calculation of the interaction relationship,

• (4) Calculate the influence weights and draw the influence relationship map.

Fig. 2

Model procedure for exploring key factors

The steps of each stage are described in detail in the following sub-sections.

3.1 Develop Goals and Define the Evaluation Framework

The first stage is divided into four steps: confirm the goal of decision-making, define the evaluation system and the expert group, define the neutrosophic sets (NSs), and obtain a semi-structured survey.

3.1.1 Step 1: Confirm the Goal of Decision-Making

In general, the response to a national public health emergency needs to consider multiple dimensions, such as prevention, detection, response, IHR-related hazards and points of entry. These dimensions have interactive influences and relationships with each other. This study explores the influence relationships within the entire public health emergency response system based on pairwise comparisons, to identify which are causal relationships affecting the entire evaluation system. The results can facilitate the evaluation of epidemic prevention decisions, to help reduce the impact and improve the prevention of the epidemic. Therefore, the main goal of this study is to explore the key factors for the national public health emergency response system.

3.1.2 Step 2: Define the Evaluation System and the Expert Group

The entire national public health emergency response decision-making evaluation system is composed of n indicators C_j (j = 1, 2,…,n), belonging to m dimensions D_j (j = 1, 2,…,m). The decision group is composed of k experts k_i (i = 1, 2,…,k).

3.1.3 Step 3: Define the Neutrosophic Set (NSs)

Let X be a space consisting of generic elements represented by x. An neutrosophic set Q can be denoted by $Q = \left\{ {\left[ {x,\;T_{Q} (x),\;I_{Q} (x),\;F_{Q} (x)} \right]\;\left| {x \in X} \right.} \right\}$; this parameter is different from the membership functions proposed by Zadeh. For the difference of membership function between detailed Fuzzy theory and Neutrosophic theory, please refer to the diagram of Tiwari and Kumar [67], as shown in “Appendix” Fig. 6. It is worth noting that from this figure we can inferred that the singled value Neutrosophic set is the generalization of all fuzzy theories.

$T_{Q} (x)$ represents the truth-membership function, $I_{Q} (x)$ represents the indeterminacy-membership function, and $F_{Q} (x)$ represents the falsity-membership function. The three functions are standard subsets as in $T_{Q} (x),\;I_{Q} (x),\;F_{Q} (x) \to \left[ {0,\;1} \right]$, and the sum of the three functions will be between 0 and 3, which is defined as $0 \le T_{Q} (x) + I_{Q} (x) + F_{Q} (x) \le 3$.

3.1.4 Step 4: Obtain a Semi-structured Interview Questionnaire

This study will conduct decision-making analysis based on the influence relationship between the evaluation systems, and draw up a semi-structured questionnaire based on the evaluation framework. However, these influence relationships are difficult to obtain from quantitative data. Therefore, the technique of qualitative analysis is adopted and the influence relationship between indicators confirmed by relying on the expert system. The outline of the interview and the items to be discussed will be designed before the interviews.

3.2 Evaluation of Incomplete, Uncertain and Inconsistent Information

The second stage is divided into four steps: collection and coding of expert opinions, the neutrosophic set matrix $\widetilde{{\varvec{E}}}_{i}$ of each expert, the deneutrosophication matrix ${\varvec{B}}_{i}$ of each expert, and the average direct influence matrix ${\varvec{A}}$.

3.2.1 Step 1: Collecting and Coding of Expert Opinions

During the interview process, semantic corrections are confirmed with experts in advance, and the purpose of the entire study explained and communicated with them. The degree of influence is rated on the following scale: No Influence, Very Low Influence, Low Influence, High Influence and Very High Influence. After obtaining the explicit value matrix of the influence relationship between each comparison, we use single-valued neutrosophic numbers (SVNN) to transform the matrix, as shown in Table 1.

Table 1 Semantic, crisp values and corresponding single-valued neutrosophic numbers

3.2.2 Step 2: Obtain the Neutrosophic Set Matrix $\widetilde{{\varvec{E}}}_{i}$ for Each Expert

By encoding the semi-structured questionnaire, we can obtain the expert's initial explicit value evaluation matrix ${\varvec{V}}$. The equation for the criterion is shown in Eq. (1); the equation for the dimension is similar to that of the criterion.

$$ {\varvec{V}} = \left[ {v_{ij} } \right]_{n \times n} = \begin{array}{*{20}c} {\begin{array}{*{20}c} {} & {\begin{array}{*{20}c} {C_{1} } & \cdots & {C_{j} } & \cdots & {C_{n} } \\ \end{array} } \\ \end{array} } \\ {\begin{array}{*{20}c} {\begin{array}{*{20}c} {C_{1} } \\ \vdots \\ {C_{i} } \\ \vdots \\ {C_{n} } \\ \end{array} } & {\left[ {\begin{array}{*{20}c} {v_{11} } & \cdots & {v_{1j} } & \cdots & {v_{1n} } \\ \vdots & {} & \vdots & {} & \vdots \\ {v_{i1} } & \cdots & {v_{ij} } & \cdots & {v_{in} } \\ \vdots & {} & \vdots & {} & \vdots \\ {v_{n1} } & \cdots & {v_{nj} } & \cdots & {v_{nn} } \\ \end{array} } \right]} \\ \end{array} } \\ \end{array}_{n \times n} . $$

(1)

After obtaining the initial explicit value evaluation matrix ${\varvec{V}}$, we convert the matrix according to the single-valued neutrosophic numbers corresponding to Table 1 to obtain the neutrosophic set matrix $\widetilde{{\varvec{E}}}_{i}$ for every expert, as shown in Eq. (2).

$$ \widetilde{E}_{i} = (E_{i}^{T} ,\;E_{i}^{I} ,\;E_{i}^{F} ). $$

(2)

3.2.3 Step 3: Obtain the Deneutrosophication Matrix ${\varvec{B}}_{i}$ for Each Expert

Use the Eq. (3) for deneutrosophication, we are able to get the deneutrosophication matrix ${\varvec{B}}_{i}$ for each expert. This step is similar to traditional defuzzification, and finally we obtain an explicit value from the neutrosophic number.

$$ {\varvec{B}}_{i} = 1 - \sqrt {\left\{ {\left[ {1 - T(x)} \right]^{2} + \left[ {I(x)} \right]^{2} + \left[ {F(x)} \right]^{2} } \right\}/3} . $$

(3)

3.2.4 Step 4: Obtain the Average Direct Influence Matrix ${\varvec{A}}$

The evaluation using neutrosophic sets can effectively and comprehensively consider incomplete, uncertain and inconsistent information. Since the decision-making process is reliant upon the decisions made by the expert groups, therefore, we integrate the opinions of experts as a measure of the central tendency. Finally, the average direct influence relationship matrix ${\varvec{A}}$ will be obtained as in Eq. (4).

$$ {\varvec{A}} = \left[ {a_{ij} } \right]_{n \times n} = \begin{array}{*{20}c} {\begin{array}{*{20}c} {} & {\begin{array}{*{20}c} {C_{1} } & \cdots & {C_{j} } & \cdots & {C_{n} } \\ \end{array} } \\ \end{array} } \\ {\begin{array}{*{20}c} {\begin{array}{*{20}c} {C_{1} } \\ \vdots \\ {C_{i} } \\ \vdots \\ {C_{n} } \\ \end{array} } & {\left[ {\begin{array}{*{20}c} {a_{11} } & \cdots & {a_{1j} } & \cdots & {a_{1n} } \\ \vdots & {} & \vdots & {} & \vdots \\ {a_{i1} } & \cdots & {a_{ij} } & \cdots & {a_{in} } \\ \vdots & {} & \vdots & {} & \vdots \\ {a_{n1} } & \cdots & {a_{nj} } & \cdots & {a_{nn} } \\ \end{array} } \right]} \\ \end{array} } \\ \end{array}_{n \times n} . $$

(4)

3.3 Calculation of the Interaction Relationship

The third stage uses the output of the second stage to calculate the interaction relationship. The third stage is divided into four steps:

3.3.1 Step 1: Obtain the Normalized Average Clarity Direct Relation Matrix, ${\varvec{N}}$

where u is a constant used for normalizing ${\varvec{N}}$ as in Eq. (5).

Here u is a constant used for normalizing ${\varvec{N}}$ as in Eq. (6).

$$ u = \max \left[ {\mathop {\max }\limits_{1 \le i \le n} \sum\limits_{j = 1}^{n} {a_{ij} ,} \;\mathop {\max }\limits_{1 \le j \le n} \sum\limits_{i = 1}^{n} {a_{ij} } } \right]. $$

(5)

Matrix ${\varvec{N}}$ can be derived from Eq. (6).

$$ {\varvec{N}} = \left[ {n_{ij} } \right]_{n \times n} = {\varvec{E}}/u. $$

(6)

3.3.2 Step 2: Derive the Total Influence Relation Matrix T

The indirect effects of the model diminish with an increase in the power of ${\varvec{N}}$. The total influence relation matrix ${\varvec{T}}$ is defined by Eqs. (7) and (9), in which ${\varvec{I}}$ denotes the identity matrix. If at least one row or column of summation is equal to 1 (but not all) in $\sum\nolimits_{j = 1}^{n} {t_{ij} }$ and $\sum\nolimits_{i = 1}^{n} {t_{ij} } ,$ then we can guarantee $\mathop {\lim }\limits_{k \to \infty } {\varvec{N}}^{g} = [0]$$_{n \times n}[0]$$_{n \times n}$

$$ \begin{aligned} {\varvec{T}} & = {\varvec{N}} + {\varvec{N}}^{2} + \cdots + {\varvec{N}}^{g} \\ & = {\varvec{N}}({\varvec{I}} + {\varvec{N}} + \cdots + {\varvec{N}}^{g - 1} )[({\varvec{I}} - {\varvec{N}})({\varvec{I}} - {\varvec{N}})^{ - 1} ] \\ & = {\varvec{N}}({\varvec{I}} - {\varvec{N}}^{g} )({\varvec{I}} - {\varvec{N}})^{ - 1} \;\; \\ & = {\varvec{N}}({\varvec{I}} - {\varvec{N}})^{ - 1} ,\quad {\text{when}}\;g \to \infty ,\quad {\varvec{N}}^{g} = \left[ 0 \right]_{n \times n} , \\ \end{aligned} $$

(7)

where ${\varvec{T}} = [t_{ij} ]_{n \times n} ,\;0 \le t_{ij} < 1,\;0 < \sum\nolimits_{j = 1}^{n} {t_{ij} } ,\;\sum\nolimits_{i = 1}^{n} {t_{ij} } \le 1.$

$$ {\varvec{T}} = \left[ {t_{ij} } \right]_{n \times n} = \begin{array}{*{20}c} {\begin{array}{*{20}c} {} & {\begin{array}{*{20}c} {C_{1} } & \cdots & {C_{j} } & \cdots & {C_{n} } \\ \end{array} } \\ \end{array} } \\ {\begin{array}{*{20}c} {\begin{array}{*{20}c} {C_{1} } \\ \vdots \\ {C_{i} } \\ \vdots \\ {C_{n} } \\ \end{array} } & {\left[ {\begin{array}{*{20}c} {t_{11} } & \cdots & {t_{1j} } & \cdots & {t_{1n} } \\ \vdots & {} & \vdots & {} & \vdots \\ {t_{i1} } & \cdots & {t_{ij} } & \cdots & {t_{in} } \\ \vdots & {} & \vdots & {} & \vdots \\ {t_{n1} } & \cdots & {t_{nj} } & \cdots & {t_{nn} } \\ \end{array} } \right]} \\ \end{array} } \\ \end{array}_{n \times n} , $$

(8)

$$ {\varvec{c}} = (c_{1} , \ldots ,c_{j} , \ldots ,c_{n} )^{\prime} = \left[ {c_{{_{j} }} } \right]^{\prime }_{n \times 1} = \left[ {\sum\limits_{i = 1}^{n} {t_{ij} } } \right]^{\prime }_{1 \times n} . $$

(9)

3.3.3 Step 3: Calculation of the Influence Level

In this step, we add up the degree of influence of each criterion on other criterion to obtain the degree of influence of n criteria, represented by vector ${\varvec{r}}$, as shown in Eq. (10).

$$ {\varvec{r}} = (r_{1} , \ldots ,r_{i} , \ldots ,r_{n} ) = \left[ {r_{{_{i} }} } \right]_{n \times 1} = \left[ {\sum\limits_{j = 1}^{n} {t_{ij} } } \right]_{n \times 1} . $$

(10)

By adding up the degree of influence of each criterion by other criterion, the degree of influence of n criteria can be obtained, which is represented by a vector as shown in Eq. (11).

$$ {\varvec{c}} = (c_{1} , \ldots ,c_{j} , \ldots ,c_{n} )^{\prime} = \left[ {c_{{_{j} }} } \right]^{\prime }_{n \times 1} = \left[ {\sum\limits_{i = 1}^{n} {t_{ij} } } \right]^{\prime }_{1 \times n} . $$

(11)

3.3.4 Step 4: Calculation of Prominence and Relation

The degree of influence and the degree of being influenced of each criterion are added up to obtain the total degree of influence of each criterion (regardless of whether is influences or is influenced). Vector ${\varvec{f}}$ is labeled “Prominence”, as shown in Eq. (12).

$$ {\varvec{f}} = \left[ {f_{{_{i} }} } \right]_{n \times 1} = (f_{1} , \ldots ,f_{i} , \ldots ,f_{n} ), $$

(12)

$$ f_{i} = r_{i} + c_{i} . $$

(13)

We subtract the degree of influence and the degree of being influenced for each criterion to obtain the net degree of influence of each criterion (a positive value indicates a cause, and a negative value an effect). Vector ${\varvec{z}}$ is labeled “Relation”, as shown in Eq. (14).

$$ {\varvec{z}} = \left[ {z_{{_{i} }} } \right]_{n \times 1} = (z_{1} , \ldots ,z_{i} , \ldots ,z_{n} ), $$

(14)

$$ {\varvec{z}}_{i} = r_{i} - c_{i} . $$

(15)

3.4 Calculate the Influence Weights and Draw the Influence Relationship Map

3.4.1 Step 1: Calculate the Influence Weights

For the calculation of the influence weight we must consider both the distance between Relation and the origin and Prominence and the origin. The farther the distance is, the greater the weight of the influence weight. A greater influence weight implies that the criterion is more important, as shown in Eqs. (16)–(19).

$$ {\varvec{o}} = \left[ {o_{{_{i} }} } \right]_{n \times 1} = (o_{1} , \ldots ,o_{i} , \ldots ,o_{n} ), $$

(16)

$$ o_{i} = \sqrt {z_{i}^{2} - f_{i}^{2} } , $$

(17)

$$ {\varvec{w}} = \left[ {w_{{_{i} }} } \right]_{n \times 1} = (w_{1} , \ldots ,w_{i} , \ldots ,w_{n} ), $$

(18)

$$ w_{i} = o_{i} /\sum\limits_{i = 1}^{n} {o_{i} } . $$

(19)

3.4.2 Step 2: Drawing the Influential Network Relation Map and Classification

From the fourth step of the previous stage, we obtain the Relation and Prominence between dimensions and criteria. The Relation and Prominence are then projected on the x and y axes to produce an influential network relation map of the entire evaluation system, as shown in Fig. 3. Prominence goes from left to right from low to high, with the average of the system's Prominence being in the middle. Relation represents the relationship played by the criterion in the system. The median value is “0”. A positive value represents a cause, and a negative value represents an effect. Using the median of Prominence and Relation, respectively, the entire influential network relation map can be divided into four quadrants. From upper right and moving counterclockwise to the lower right they are denoted: Quadrants I, II, III, and IV. The dimensions and criteria in the first quadrant have a stronger Prominence and are thus a cause in the entire system. In other words, the dimensions and criteria that fall in this quadrant will be the key factors affect the entire system, which we call the “core”. The dimensions and criteria in the second quadrant are also classified as causes in the system, but with less Prominence. This means that the dimensions and criteria that fall into this quadrant will have an impact on the entire system, but the impact is not strong, also called “Drivers”. The dimensions and criteria in the third quadrant are characterized as effects in the system, with weaker Prominence. Therefore, the dimensions and criteria that fall in this quadrant have the characteristic of being “Independent”. The dimensions and criteria in the fourth quadrant are characterized as effects in the system but with stronger Prominence. Therefore, although the dimensions and criteria that fall in this quadrant cannot directly improve the system, they are still needed to strengthen the management of these dimensions and criteria, which we call “Impact”.

Fig. 3

The influence classification map

4 Empirical Example

This section will describe the problem, data collection, and the evaluation under incomplete information.

4.1 Problem Description

COVID-19 has the characteristics of high infectivity, rapid spread and mutation. Given the seriousness, on January 30, 2020, the WHO declared that the new Coronavirus outbreak constitutes an “international public health emergency”. Medicine has been regarded as one of the most important comprehensive sciences in the history of human development. In biomedicine vaccines are considered as one of the most effective ways for humans to resist viruses. Numerous organizations and institutions are actively participating in the research and development, and production of a vaccine for COVID-19. As of November 2020, according to the WHO, there are 212 vaccines in the developmental stage, with 42 candidate vaccines under clinical evaluation. However, scientists and researchers around the world are forced to fight this disease in the face of a lot of incomplete, uncertain and inconsistent information. Many countries suffering from COVID-19 are trying to use quarantines or other methods to protect their citizens before the vaccines are released to the public.

It is worth noting that Taiwan and China have a high degree of close economic and commercial trade exchanges. Of Taiwan’s population of 23 million people, about 850,000 people live in China, with 404,000 employed there [25, 34]. Due to their proximity and frequency of flights, in 2019 Taiwan was predicted to have the second highest number of confirmed cases. However, as of November 16, 2020, Taiwan has had only 602 confirmed cases and only 7 deaths. The collective efforts of Taiwan's management authorities have resulted in the relatively low number of infections and deaths, and has been hailed as a successful anomaly in the global pandemic [71]. This makes Taiwan’s national public health emergency response system an interesting case to discuss. Therefore, this study will take Taiwan’s public health emergency response system as a case study to demonstrate the proposed integrated neutrosophic and DEMATEL model. The key elements that affect the entire system, with consideration of incomplete, uncertain and inconsistent information are explored, and visualized graphs used to show the influence relationships within the public health emergency response system. The results are expected to help governments and organizations understand the cause and effect relationship within the public health emergency response system, and better set prevention strategies.

4.2 Data Collection

The JEE system, as mentioned above, includes four dimensions (i.e., prevention, detection, response, other hazards). As indicated in Rezaei et al. [72] only 4–10 decision-makers are required to getter reliable data for MCDM analysis. This study relies on the judgements of experts who have many years’ experience in the public health system; see Table 2. These experts included senior practitioners in several fields related to the public health system: two medical experts with public health expertise; one public health economist; one public health information technology (IT) expert; and one public health care professional; one public health testing expert. They generally had between 25 and 30 years work experience.

Table 2 The background of six the experts

The participating experts were mainly department heads, and two of them were deputy deans. They had backgrounds consistent with the fields of public health precaution assessment, including medicine, nursing, preventive science, detection science, informatics economics, etc. Therefore, the composition of these experts has a certain degree of representativeness. Although only six experts were interviewed, the consensus index of the six questionnaires was 0.044, a confidence level of more than 95%.

After confirming the list of individuals to be interviewed experts, we conducted in-depth interviews. During the interview process we used a semi-structured interview and one-to-one questionnaire surveys. First, we explained the essential meaning of the entire evaluation system. Second, the dimensions and criteria were defined. Finally, we introduced the specific operation methods of the investigation process. To avoid omissions and make sure the completeness of information collection, we double checked the results of each answer with all the experts. Each interview took more than one hour.

Since the DEMATEL method collects data by pairwise comparison, the questionnaire will ask about the influence of indicators, respectively. As shown in “Appendix” Fig. 7. The questioning are divided into two phase.

• (1) In the first phase, we will ask whether dimensions/criteria A will affect dimensions/criteria B. If the answer is negative, “A” to “B” is coded as “0” If the answer is positive, we will move on to the next phase.

• (2) In the second phase, we will ask the degree of the impact. We divided the evaluation scale into very low, low, high, and very high. Then we will encode “A” to “B” according to the response.

4.3 Evaluation Under Incomplete Information

This study considers the impact of incomplete, uncertain and inconsistent information in the process of decision-making. The single value neutrosophic set technique is applied to represent the experts’ opinions. The detailed calculation process of this technique is shown in Eqs. (1)–(3). Due to space limitations, we will present only the evaluation results for the dimensions, as shown in Table 3.

Table 3 Evaluation matrix under incomplete information

The left side of the table represents the initial opinions collected after each expert interview. The semantic expressions of the experts were converted into corresponding values as indicated in the center. The right side of the table shows the results of the conversion operation of the neutrosophic set. The neutrosophic set conversion process uses single value neutrosophic numbers as shown in Table 1 in Sect. 3. Finally, Eq. (3) is used to obtain the deneutrosophic set.

5 Results and Discussion

This section provides a detailed description of the influence relationship, influence weights, and the influence network relationship map (INRM).

5.1 Influence relationship for a public health assessment system

According to the deneutrosophic results of the evaluation matrix, the average numbers are used to integrate the opinions of all experts and obtain the direct influence relationship matrix (Table 4). The right side of Table 4 shows the total influence relationship matrix and influence relationship of each dimension.

Table 4 Influence relationship matrix by direct and total

We use Eqs. (10)–(11) to get the influence degree (r) and the affected degree (c) for each dimension. It is found that dimension D₂ is the dimension that affects the other dimensions the most, and dimension D₄ is the dimension most affected by the other dimensions. Here, f refers to the sum of the influence degree (r) and the affected degree (c) of each dimension. If the value obtained for this dimension is larger, it means that this dimension has a greater impact on the public health emergency response system in the country. Based on the results of f, the degree of total impact for the entire national public health emergency response system is D₂ > D₃ > D₄ > D₁. Therefore, Detection (D₂) is the most prominent dimension in the national public health emergency response system. Based on net influence (z), the values of D₂ and D₁ are positive, and the values of D₃ and D₄ are negative. Therefore, D₂ and D₁ are determined to be the cause of the causality relationship, while D₃ and D₄ can be categorized as the effect of the causality relationship.

The dimension of Prevention (D₁) consists of seven criteria: national legislation, policy and financing, IHR coordination, communication and advocacy, antimicrobial resistance, zoonotic disease, food safety, biosafety and biosecurity, immunization. Tables 5 and 6 shows the degree of influence among the seven criteria in D₁.

Table 5 The degree of influence among the seven criteria in Prevention (D₁)

Table 6 Continuation of Table 5

Based on the total influence (f) in Table 6, their prominence is ranked as follows: National legislation, policy and financing (C₁; 17.26), IHR coordination, communication and advocacy (C₂; 17.01), biosafety and biosecurity (C₆; 16.69), food safety (C₅; 16.30), zoonotic disease (C₄; 16.22), immunization (C₇; 16.08), Antimicrobial resistance (C₃; 15.83).

The dimension of Detect (D₂) consists of four criteria: national laboratory system, surveillance, reporting, human resources. Table 7 shows the degrees of influence among the seven criteria in D₂. Their prominence is ranked in the following order: surveillance (C₉; 26.11), reporting (C₁₀; 25.83), national laboratory system (C₈; 25.82), human resources (C₁₁; 25.42).

Table 7 The degree of influence among the four criteria in Detect (D₂)

The dimension of Response (D₃) consists of five criteria: emergency preparedness, emergency response operations, linking public health and security authorities, medical countermeasures and personnel deployment, risk communication. Tables 8 and 9 shows the degree of influence among the four criterion in D₃ in order of prominence as follows: emergency response operations (C₁₃; 91.24), linking public health and security authorities (C₁₄; 91.24), emergency preparedness (C₁₂; 90.01), medical countermeasures and personnel deployment (C₁₅; 89.41), and risk communication (C₁₆; 87.83) (see Tables 8, 9).

Table 8 The degree of influence among the four criteria in Response (D₃)

Table 9 Continuation of Table 8

The dimension of IHR-related hazards and points of entry (D₄) consists of three criteria: points of entry, chemical events, radiation emergencies. Table 10 shows the degree of influence among the three criterion in D₄. The prominence is ranked in the order of Radiation emergencies (C₁₉; 27.37), Points of entry (C₁₇; 27.26), Chemical events (C₁₈; 25.85).

Table 10 The degree of influence among the three criteria in IHR-related hazards and points of entry (D₄)

5.2 Influence Weights for the Public Health Assessment System

The DEMATEL technique was applied to obtain the influence weight between dimensions and criteria. The larger the influence weight, the more important the dimension/criterion will be for the evaluation system. The influence weights were calculated differently than in previous research [71], which only calculated the weight based on the “Prominence”. We considered the distance from the Relation to the origin and the distance from the Prominence to the origin at the same time [73]. Table 11 shows the influence weights of the 19 criteria and 4 dimensions in the public health emergency response system.

Table 11 Influence weights for a public health assessment system

When we view the entire public health emergency response system with these 19 criteria, the order of the influence weight of each criterion is:

$$ C_{19} > C_{17} > C_{18} > C_{9} > C_{8} > C_{10} > C_{11} > C_{13} > C_{12} > C_{15} > C_{14} > C_{16} > C_{1} > C_{2} > C_{6} > C_{5} > C_{4} > C_{7} > C_{3} . $$

When we view the entire public health emergency response system with these four dimensions, the influence weight of each dimension is: Detection (0.26), Response (0.25), IHR-related hazards and points of entry (0.25), Prevention (0.24), from a total of 1.000. Based on the above results, we can see that “Detection” is a relatively important dimension in the public health emergency response system. However, it is worth noting that the gap between the top three dimensions is actually not that large. It can be seen that the management of the public health emergency response system must be comprehensive and balanced in all aspects. Flaws and breaches are not allowed in public health management. Therefore, we recommend that each management authority should upgrade the entire system given limited resources. The public health emergency response system can allocate-related resources according to the corresponding weights, taking comprehensive and balanced development as the primary management strategy.

In addition, we can discuss different criteria according to the corresponding dimensions. For example, in the dimension “Detect” the top three rankings are surveillance (0.25), national laboratory system (0.25), and reporting (0.25). In the dimension “Response,” the top three in the ranking are emergency response operations (0.20), emergency preparedness (0.20), and medical countermeasures and personnel deployment (0.20).

5.3 Influence Network Relationship Map (INRM)

Following the steps of Sect. 3, we calculate the average of prominence and divide the INRM into four quadrants. The first quadrant contains the core factors, the second quadrant includes the driving factors, the third quadrant includes the independent factors, and the fourth quadrant comprise the influential factors. The INRM for the public health emergency response system is shown in Fig. 4.

Fig. 4

INRM of the public health assessment system (Main)

Decision makers can intuitively observe the complex causal relationship between indicators and further provide valuable insights for decision-making. “Detection” plays a crucial role in the entire public health emergency response system, and “Prevention” is the driving factor. “Response” and “IHR related hazards and points of entry” are the influencing factors.

From Fig. 5 we can observe the complex causal relationship between criteria in each dimension. “Reporting” is the core criterion in the dimension “Detection”, “human resources” is the driving factor, and “national laboratory system and surveillance” is the influential factor in this dimension.

Fig. 5

INRM of Public health assessment system

“National legislation, policy and financing and IHR coordination, communication and advocacy” are the core criteria in the dimension “Prevention”, “zoonotic disease” is the driving factor, “biosafety and biosecurity” is the influencing factor, and “antimicrobial resistance, food safety and Immunization” are independent factor sin this dimension.

In the dimension “Response,” “emergency preparedness and emergency response operations” are the core criteria, “risk communication” is the influencing factor, “linking public health and security authorities” and “medical countermeasures and personnel deployment” are independent factors.

Based on the above findings, “Detection” in the national public health emergency response system is the key factor affecting the entire system, and “Reporting” is the essential criterion in the dimension “Detection”. In the dimension “Detection” “human resources” will drive improvement. Although “national laboratory system and surveillance” cannot directly improve the dimension, we still need to strengthen its management.

6 Discussion

Based on the results of the analysis in the previous chapter, this chapter will carry out a comprehensive discussion, including managerial implications and theoretical implications.

6.1 Managerial Implications

The results of this study are similar to those described in previous studies [71]. First, we will advise the authorities of the public health emergency response system of various countries to actively face the current situation when faced with a public health emergency, and report to the relevant international agencies. A complete notification system and a friendly internal notification mechanism should set up within the public health system. The competent authorities should actively and publicly release information about the status of public health emergencies, and make relevant explanations to remove all doubts.

Secondly, we can see from the INRM that “human resources” has a low influence, but it is the main driver in this dimension. Although, one may not see an obviously direct impact in the short term, as long as human resources are improved, it will lead to improvement of the entire system. Due to the unknown and highly infectious nature of COVID-19, the protection of front-line personnel is also very important, and will promote the improvement of the “Detection” system. To this end, we recommend that the authorities formulate relevant policies and measures to give priority to the protection of frontline personnel. Before developing vaccines, scientists have to first develop a quick and convenient tool to detect the virus. In addition to the protection provided by the emergency response system to the front-line personnel, we should also focus on the maintenance of human resources in the medical industry. Meskó et al. [74] pointed out that in the global healthcare industry, human resource is facing three main crises: a global shortage of doctors, the aging and exhaustion of doctors, and the increasing demand for long-term care. Human resources in Taiwan's healthcare industry also face similar problems. In the past few years, the medical profession in Taiwan has suffered from the “Five Big Losses”, the lack of freshman entering the five major departments of internal medicine, external medicine, maternity, pediatrics, and emergency department along with long working hours, low health insurance benefits, high surgical risks, and many medical disputes. This is very worrying [75]. If the human resources in the healthcare industry are often overloaded and on the verge of collapse, it will be very easy for unexpected challenges such as the COVID-19 pandemic to drive the emergency response system over the edge. Therefore, we suggest that the relevant government agencies can move towards long-term management for the protection of frontline personnel and human resources in the medical industry, formulating strategies in order to prevent the pandemic from spreading further. For example, provide reasonable subsidies to medical staff during the epidemic; strengthen epidemic-related training; and recruit the services of overseas doctors encouraging them to return home.

Finally, we need to strengthen the management of the “national laboratory system and surveillance.” The results suggest that the authorities can input relevant resources to assist scientific research institutions rapidly integrate research and strengthen the sharing mechanism. Due to the national public emergency response system being interactive, as long as we can improve the root causes of the system, the entire system will comprehensively improve. In the face of the unknown impact of COVID-19 and resource limitations, we recommend allocating resources based on the weights given in Table 9. In addition, during this period, we can establish a dedicated agency and dispatch relevant information to each agency to form a project team, so that experts can utilize their strengths to improve the effectiveness of the surveillance system.

6.2 Theoretical Implications

In the past, multiple methods have been used to explore key factors. DEMATEL is one of the most frequently used methods, mainly because of its ease of use and its flexibility for integration with a variety of other methods. Compared with other methods, it is not limited by the assumption of independence and considers the interaction between indicators inside the evaluation systems. Another of the advantages of the DEMATEL is the visual analysis producing results which are helpful for analysis and decision-making.

However, the DEMATEL technique cannot effectively handle the divergence of natural language semantics. Fuzzy theory has often been used to extend the DEMATEL technique; the so-called Fuzzy DEMATEL. However, these extensions do not entirely focus on measurement of the ambiguity and inconsistent information. The neutrosophic set theory was mainly developed to measure incomplete information. Therefore, it is suitable to integrate the DEMATEL technique with neutrosophic set theory as a way to compensate for deficiencies in the DEMATEL technique’s ability to measure the inconsistent information in the decision-making environment and to explore the key factors for COVID-19 prevention. Experts are also human beings whose opinions are expressed in natural language semantics which are inherently divergent. The gap of such discordance will become bigger under incomplete information. Therefore, applying the proposed approach to COVID-19 prevention decision-making, a situation which is full of unknowns, could yield substantial theoretical benefits.

6.3 Comparative Analysis

In order to illustrate the differences between the traditional DEMATEL technique and the one which is combined with fuzzy theory, we carry out a comparative analysis in this study, as shown in Table 12. We can see that the empirical results change depending upon the method used. The order of total influence relationship analyzed by the traditional method is D₂ > D₄ > D₃ > D₁, while the result of analysis by the integrated method is D₂ > D₃ > D₄ > D₁. In terms of causality, the empirical results for dimension D₃ are also different. We can see that the results produced by the integrated method are different from those produced by the traditional method. Incomplete information apparently affects the outcome of the decision-making analysis, and the proposed method is useful for measuring incomplete information to a certain extent.

Table 12 Comparison of traditional and integrated methods

In addition, it should be noted that expert opinions can be integrated in many different ways. Two scenarios are given as an example to calculate the weight of the experts’ opinion for analysis and comparison, as shown in Table 13. In the first scenario, the weights of the expert opinions are calculated based on the level of education of the experts. The higher the education level, the higher the assigned weight. Here, “E” represents the weight. In the other scenario, the weight is calculated with consideration of both the education level and seniority of the expert. The higher the education level and seniority, the higher the weight, which is represented by “E & S”. The latter method is proposed for use in this study.

Table 13 Compare of traditional and integrated methods

We used the expert weights shown in Table 13 for integration of the experts’ opinions, and the results of the analysis are shown in Table 14. As can be seen in Table 14, although the coefficients of the influence relationship are different, there is no obvious difference between the order of the total influence relationship and the causality of the criteria. Therefore, after confirming that the experts had reached a consensus on the questions from the questionnaire through the consensus degree test, the opinions were integrated by adopting an arithmetic average method.

Table 14 Comparison of integrated opinions

Based on the above comparative analysis, we can say that the proposed method is very suitable for measuring incomplete information, and the proposed method is more suitable and effective for real world decision-making than the traditional method.

7 Conclusions

Currently, the survival and development of human civilization has been severely impacted by COVID-19. In the face of this unknown infectious disease, how to respond, prevent and reduce the impact and loss will be a crucial issue to humans. When it comes to the unknown decision-making process, how to effectively consider these incomplete, uncertain and inconsistent information will be a major challenge. This study combines Neutrosophic set with DEMATEL method and use the JEE to explore the key factors for preventing epidemics under the information uncertainty. The study found that among the four dimensions, “Detection” is the key to affect the entire system. “Prevent” drives the entire evaluation system. The management of Response and IHR-related hazards and Points of entry needs to be improved. It is worth noting that “reporting,” national legislation, policy and financing,” “IHR coordination, communication and advocacy,” “emergency preparedness,” and “emergency response operations” are the essential criteria affecting each aspect. Since “Detection” is the source of the impact, and “reporting” is the source of the impact on the dimension “Detection” therefore, we suggest that when the management authorities of various countries are striving to prevent an unknown epidemic, they should strengthen international and foreign, internal and external notification systems, disclose and report true data and current conditions.

Although this research is expected to make some contribution to improve the national public health emergency response system, there are still some suggestions for future work. First, to explore the national public health emergency response system from a different perspective. Different evaluation frameworks might lead to different results and management implications. Second, to evaluate the performance of the public health emergency response systems of various countries. The current study offers only a snapshot of Taiwan and comparisons with our study are welcomed. The results only consider the average of the experts’ judgements. Therefore, in the future, it may be possible to use different technologies to merge the opinions of the group. For example, by giving different experts different weights or making decisions with the concept of intervals might be possible ways.

Appendix

The schematic diagram that the “Appendix” will is from Tiwari and Kumar [67] and the DEMATEL technique questionnaire interview (see Figs. 6 , 7 ).

Fig. 6

The diagram of membership function of Fuzzy theory and Neutrosophic theory

Fig. 7

The process of DEMATEL questionnaire