1 Introduction

Railway infrastructure maintenance is essential to ensure normal operation of railway systems. Both train operation and maintenance work use railway infrastructure, such that when a train is running, maintenance work is not allowed. To solve this problem, railway companies will reserve slots in the timetable, called maintenance window (MW), which have three basic elements: type, time length, and location in the timetable. For a particular railway line, there are a variety of scenarios for setting MW, each of which is referred to as a MWS.

At present, MWS setting is a challenging planning problem for railways. On the one hand, MWS setting must consider the requirements of many different departments, including the works department, electricity department, and signal department. Their requirements are often different and sometimes conflicting. Coordinating conflicting demands between different departments is a complicated problem. On the other hand, there is an interrelation between MWS setting and train timetable planning. Drawing of train paths will limit the time range of the MWS setting, and MWS setting also reduces train operation time and divides the timetable into two separated periods, which has a large impact on railway transportation capacity. Thus, reasonable MWS is needed to ensure safe railway operation and improve railway capacity.

In essence, MWS setting is a multi-objective programming problem whose complexity has been acknowledged [1,2,3,4]. Considering the complexity of this problem, most countries still use a computer-aided decision system operated by experienced planners to set the MWS. Because a computer can assist planners in designing a number of MWSs within an acceptable period of time and different personnel design schemes, it is important to evaluate the advantages and disadvantages of multiple MWSs and choose the best one.

Evaluation of MWS setting is a relatively new topic in the field of railway operation research. Several different indicators have been proposed to evaluate MWS in a few studies. Lidén [5] used maintenance cost and expected train traffic demand as a measure of MWS in Sweden. Meng [6] used occupancy time of MWS, the influence of MWS on trains, the safety of maintenance, and other factors to formulate an evaluation indicator system in China. Xiang et al. [7] constructed a selection evaluation model to determine reasonable MW locations in the timetable, using the number of affected trains, types of affected trains, and the nature of the conflict as indicators. Scholars have used fuzzy comprehensive evaluation [7, 8], entropy [6], and other uncertainty theories [9] for evaluation. It is worth mentioning that Lidén [5] proposed a MWS evaluation method based on cost measurement. However, application of this method is difficult because it requires complex calculations and large amounts of data.

On the whole, the existing research on MWS evaluation is inadequate for a number of reasons. First, few studies have focused on MWS evaluation, resulting in failure to form a comprehensive and rational MWS evaluation indicator system. Second, fuzzy comprehensive evaluation, AHP, and other commonly researched methods require each evaluation expert to consider the same set of evaluation indicators and give evaluation information for each MWS. However, evaluation experts often come from different areas, or from different organizations and departments, and each expert has different knowledge and experience [10]. Therefore, each evaluation expert may only focus on a few interesting or familiar indicators. This research addresses this gap.

In this paper, we use China’s railway MWS setting as an example, achieving two important contributions. First, this paper constructs a MWS evaluation indicator system that considers the demands of different departments. Second, the fuzzy soft set is applied to MWS evaluation for the first time. Each expert utilizes a set of indicators based on personal preference during evaluation, and fuzzy soft set theory is used to integrate information obtained via expert evaluation to produce an overall evaluation.

The remainder of this paper is organized as follows: Sect. 2 establishes the MWS evaluation indicator system, Sect. 3 explains the application of fuzzy soft set theory for MWS evaluation, Sect. 4 presents a case study to verify the proposed method, and Sect. 5 offers conclusions.

2 MWS evaluation indicator system

Construction of evaluation indicator systems is important for MWS evaluation. Because MWS setting involves a number of departments such as transportation, maintenance, and dispatch, it is necessary to consider the needs of various departments when constructing the indicator system. In this section, we use a face-to-face survey method to obtain different departments’ preference for evaluation indicators. Then, we integrate them to obtain the final evaluation indicator system for MWS setting. In addition, we adopt the paired comparison method [6] to calculate the weight of each indicator.

2.1 Evaluation indicators based on different departments’ preferences

In the following, we individually analyse departments’ preferences for MWS setting and propose corresponding indicators.

  1. 1.

    Evaluation indicators based on preferences of transport department

The transport department is primarily concerned with the influence of MWS setting on the train timetable, with the goal of keeping the influence as small as possible. The indicators used to characterize the impact of the MWS setting on the train timetable drawing are mainly MW type, MW location in the timetable, influence on railway carrying capacity, influence on train travel speed, and influence on cross-line trains. Descriptions of these indicators are given in Table 1.

Table 1 Evaluation indicators based on preferences of the transport department
Full size table
  1. 2.

    Evaluation indicators based on preferences of maintenance department

Maintenance departments primarily include the public works department, the electrical department, and the signal department. These departments are primarily concerned with guaranteeing MW time and ensuring that MW setting can facilitate collaborative maintenance work between different departments. Thus, the indicators that these departments are concerned with mainly include maintenance environment safety, parallel operation adaptability, MW coordination of adjacent lines, and convenience of maintenance time. Descriptions of these indicators are given in Table 2.

Table 2 Evaluation indicators based on preferences of the maintenance department
Full size table
  1. 3.

    Evaluation indicators based on preferences of the dispatch department

Due to unexpected events, trains often deviate from the timetable. In many cases, the dispatch department will adjust the MW time to bring the train back to normal operation as soon as possible. The dispatch department therefore wants MW time to be as long and flexible as possible. MWS setting indicators for the dispatch department therefore include parallel operation adaptability of different departments, MW coordination in adjacent sections, MW coordination of adjacent lines, and difficulty of changing the MWS. Detailed descriptions of these indicators are given in Table 3.

Table 3 Evaluation indicators based on preferences of the dispatch department
Full size table
  1. 4.

    Evaluation indicators based on other departments’ preferences

The above departments are most closely related to MWS setting. However, other railway departments also may be relevant to MWS setting. For example, the statistics department will collect the basic parameters for MWS setting, which can be used as a reference for the next MWS setting. Detailed descriptions of these indicators are given in Table 4.

Table 4 Evaluation indicators based on preferences of the transport department
Full size table

2.2 Integration of evaluation indicators

The evaluation indicators system established in this paper that considers the preferences of different departments contains too many indicators, and some indicators are related to each other. So we need to eliminate redundant indicators and associated indicators. However, the commonly used index reduction methods such as principal component analysis, factor analysis, and cluster analysis require a large amount of statistical data. But most MWS evaluation indicators lack related data. More troublesome is that there are both qualitative and quantitative indicators in the proposed evaluation indicators system. These characteristics make the existing index reduction method not applicable to this paper. Therefore, we use expert group decision-making method to integrate the indicators. Based on expert knowledge and engineering judgements, these indicators primarily fall into the five following categories: basic MWS indicators, MWS setting difficulty, influence of MWS on transportation organization, coordination of MWS implementation, and safety of MWS implementation. This allows us to obtain an integrated MWS evaluation indicator system (Fig. 1). When evaluating MWS setting, evaluation experts can give the integrated indicator values based on preference indicators. Qualitative indicator values can be calculated as given in Sect. 3.1, and quantitative indicators can be calculated as given in “Appendix 1”.

Fig. 1
figure 1

MWS evaluation indicator system

Full size image

2.3 Calculation of indicator weight

The importance of different indicators in the MWS evaluation process is different, and we can assign different weights to indicators based on their influences. Determination of weights is a complex task, and in this paper, we use the pair-wise comparison matrix method. This method combines experts’ opinions and paired matrices to obtain the indicator weight. The process is as follows:

  1. 1.

    Indicator standardization

MWS evaluation indicators may have different dimensions, requiring indicators’ dimensions to be standardized.

  1. 2.

    Establishment of pair-wise comparison matrix

Weight calculations rely on pair-wise comparison matrices established through comparisons of inner-group elements. A pair-wise comparison matrix can be expressed as

$$\varvec{A} = \left[ {a_{ij} } \right]_{n \times n} \quad i,j = 1,2, \ldots ,n,$$

where \(\varvec{A}\) is the pair-wise comparison matrix; \(a_{ij}\) is an element of the matrix; \(n\) is the number of all sub-indicators. In addition, \(a_{ij}\) satisfies the following conditions:

$$a_{ij} > 0;\,a_{ij} * a_{ji} = 1;\,a_{ij} = {{a_{ik} } \mathord{\left/ {\vphantom {{a_{ik} } {a_{jk} }}} \right. \kern-0pt} {a_{jk} }}\quad i,j,k = 1,2, \ldots ,n.$$
  1. 3.

    Calculation of weights

After getting the pair-wise comparison matrix \(\varvec{A}\), we can calculate the weights of different indicators according to Eq. (1):

$$W_{i} = \, {{\sum\limits_{j = 1}^{n} {a_{ij} } } \mathord{\left/ {\vphantom {{\sum\limits_{j = 1}^{n} {a_{ij} } } {\sum\limits_{j = 1}^{n} {\sum\limits_{i = 1}^{n} {a_{ij} } } }}} \right. \kern-0pt} {\sum\limits_{j = 1}^{n} {\sum\limits_{i = 1}^{n} {a_{ij} } } }}.$$
(1)

3 Application of fuzzy soft set theory to MWS evaluation

MWS setting involves different railway departments and is very complicated. When evaluating the MWS, the opinions of experts from different departments enhance the credibility of evaluation results. However, evaluation experts from different departments may only focus on some of the indicators that they are familiar with. If evaluation experts are required to evaluate all indicators, mistakes are likely, resulting in large differences in evaluation results. Fuzzy soft set theory [11] can handle this situation effectively. In this section, we first introduce the basic theory of fuzzy soft set theory and then give the main steps of using fuzzy soft set theory to evaluate MWS setting.

3.1 Preliminaries

In this subsection, we present the basic definitions and results of soft set theory that will be used in subsequent discussion. Soft set theory was first proposed by Molodtsov in 1999 and is commonly used as a mathematical tool for analysis of uncertain objects [12], fuzzy objects [13], or objects that cannot be precisely defined [14, 15]. Soft set theory is often applied to decision-making, evaluation, attribute reduction, soft algebra, prediction, and classification.

Definition 1

(Soft set) Let \(U\) be the initial domain, \(E\) be the parameter set, and \(P(U)\) be the power set of \(U\). \((F,E)\) is a soft set on \(U\) if and only if \(F\) is a mapping of \(E\) to \(P(U)\). \(\forall e \in E\), and \(F(e)\) is the set of elements in \(U\) that exhibit the properties of attribute \(e\), that is, \(F(e) \in U\). A soft set \((F,E)\) is an approximate set composed of multiple sets, each of which exhibits the properties of the elements in \(U\).

Definition 2

(Fuzzy soft set) Let \(U\) be the initial domain, \(E\) be the parameter set, \(\xi (U)\) be a fuzzy set defined on \(U\), and \(A \subseteq E\). Then, \((F,A)\) is a fuzzy soft set on domain \(U\) if and only if \(F\) is a mapping of \(A\) to \(\xi (U)\).

Definition 3

(“AND” operation) Let \((F,A)\) and \((G,B)\) be two fuzzy soft sets on \(U\). If \(\forall (\alpha ,\beta ) \in (A,B)\), \(H(\alpha ,\beta ) = F(\alpha ) \cap G(\beta )\), then \((F,A) \wedge (G,B) =\)\((H,A \times B)\) represents the “AND” operation of \((F,A)\) and \((G,B)\).

3.2 Problem description of MWS evaluation

The formal description of the evaluation of railway MWS is as follows: Let \(M = \left\{ {m_{1} ,m_{2} , \ldots ,m_{n} } \right\}\) denote a set of MWS to be evaluated, \(m_{i}\)\((i = 1,2, \ldots ,n)\) denote the ith MWS; \(C = \left\{ {c_{1} ,c_{2} , \ldots ,c_{m} } \right\}\) is the set of evaluation indicators; and \(EX = \left\{ {ex_{1} ,ex_{2} , \ldots ,ex_{p} } \right\}\) represents the set of experts; \(w_{j}^{i}\)\((j \in \left\{ {1,2, \ldots ,m} \right\})\) represents the weight of each indicator of MWS, where \(0 \le w_{j}^{i} \le 1\) and \(\forall i \in \left\{ {1,2, \ldots ,n} \right\}\),\(\sum\limits_{j} {w_{j}^{i} } = 1\). Considering differences in knowledge, experience, and perspectives of the various experts, the evaluation indicator set of expert \(ex_{k} (k = 1,2, \ldots ,p)\) is denoted by \(C_{k} = \left\{ {c_{1}^{k} ,c_{{_{2} }}^{k} , \ldots ,c_{{l_{k} }}^{k} } \right\}\), where \(c_{j}^{k} \subseteq C\) \((j = 1,2, \ldots ,l_{k} )\),\(l_{k} \le m\). The evaluation matrix of expert \(ex_{k}\) on evaluation indicator set \(C_{k}\) for the MWS set \(M\) is \(\varvec{V}_{k} = (v_{ij}^{k} )_{{n \times l_{k} }}\), where \(v_{ij}^{k}\) denotes the value given by the expert to indicator \(c_{j}^{k}\) for \(m_{i}\).

$$\varvec{V}_{k} = \begin{array}{*{20}c} {\begin{array}{*{20}c} {} \\ {m_{1} } \\ {m_{2} } \\ \vdots \\ {m_{n} } \\ \end{array} } & {\begin{array}{*{20}c} {\begin{array}{*{20}c} {c_{1}^{k} } & {c_{2}^{k} } & \cdots & {c_{{l_{k} }}^{k} } \\ \end{array} } \\ {\left( {\begin{array}{*{20}c} {v_{11}^{k} } & {v_{12}^{k} } & \cdots & {v_{{1l_{k} }}^{k} } \\ {v_{21}^{k} } & {v_{22}^{k} } & \cdots & {v_{{2l_{k} }}^{k} } \\ \vdots & \vdots & \vdots & \vdots \\ {v_{n1}^{k} } & {v_{n2}^{k} } & \cdots & {v_{{nl_{k} }}^{k} } \\ \end{array} } \right)} \\ \end{array} } \\ \end{array} .$$
(2)

MWS evaluation indicators include both quantitative and qualitative indicators. The dimension of different indicators is different, so normalization is necessary. However, certain indicators are difficult to describe as qualitative or quantitative, such as MW coordination in adjacent sections. Simple affirmation or negation cannot accurately express expert opinions, while phrases such as “very good”, “good”, and “fair” are useful, but ambiguous.

In this study, all indicators are set with an indicator parameter, whose meaning is a certain state description of the indicator. For example, for the indicator “single MW time duration”, we can set the parameters to be “single MW time duration is reasonable”. For the indicator “the coordination between MWS and rail line conditions”, we can set the parameters as “high coordination between MWS and rail line conditions”. The expert evaluations of railway MWS are scaled as \(H = \left\{ {0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0} \right\}\). We then have \(v_{ij}^{k} \in H\), where the closer the \(v_{ij}^{k}\) value is to 1, the more likely the evaluation expert \(ex_{k}\) thinks \(m_{i}\) is consistent with the target parameters \(c_{j}^{k}\). On the other hand, the closer the value of \(v_{ij}^{k}\) is to 0, the more likely the evaluation expert \(ex_{k}\) considers \(m_{i}\) not to conform to the target parameters \(c_{j}^{k}\).

3.3 Evaluation procedure

When the MWS evaluation set and MWS evaluation index system are given, each evaluation expert gives a personal evaluation index set and an evaluation matrix. Then, the fuzzy soft set theory is used to fuse the evaluation value of all evaluation experts. We can then obtain the final evaluation results of various MWS. The detailed evaluation process is shown in Fig. 2, which contains the following three key components:

Fig. 2
figure 2

Flowchart of MWS evaluation process using fuzzy soft set theory

Full size image
  1. 1.

    Conversion of evaluation matrix and fuzzy soft set

Based on the evaluation indicator set \(C_{k}\) and the evaluation matrix \(\varvec{V}_{k}\) of each expert \(ex_{k}\), evaluations are converted into a fuzzy soft set \((F_{k} ,C_{k} )\) according to the following equations:

$$\left( {F_{k} ,C_{k} } \right) = \begin{array}{*{20}c} {\left\{ {c_{1}^{k} = \left\{ {m_{1} /v_{11}^{k} ,m_{2} /v_{21}^{k} , \cdots ,} \right.} \right.\left. {m_{n} /v_{n1}^{k} } \right\},} \\ {c_{2}^{k} = \left\{ {m_{1} /v_{12}^{k} ,} \right.m_{2} /v_{22}^{k} , \cdots ,\left. {m_{n} /v_{n2}^{k} } \right\},} \\ \cdots \\ { \, \left. {c_{{l_{k} }}^{k} = \left\{ {m_{1} /v_{{1l_{k} }}^{k} ,} \right.m_{2} /v_{{2l_{k} }}^{k} , \cdots ,\left. {m_{n} /v_{{nl_{k} }}^{k} } \right\}} \right\}.} \\ \end{array}$$
(3)
  1. 2.

    Information integration

The comprehensive evaluation matrix can be obtained after the evaluation information of each expert is fused through “AND” operations. We denote the calculation result as (G, E):

$$\left( {G,E} \right) = (G,C_{1} \times C_{2} \times \cdots \times C_{p} ) = (F_{1} ,C_{1} ) \wedge \left( {F_{2} \, ,C_{2} } \right) \wedge \cdots \wedge \left( { \, F_{p} , \, C_{p} } \right) . { }$$
(4)

Obviously, (G, E) is also a fuzzy soft set, with the parameters composed by the evaluation indicator sets C1, C2, …,Cp of the p experts. Assume that (G, E) consists of L parameters, and let \(E = \left\{ {e_{1} ,} \right.e_{2} , \ldots ,\left. {e_{L} } \right\}\). Then, (G, E) can be expressed as

$$\left( {G,E} \right) = \begin{array}{*{20}c} {\left\{ {e_{1} = \left\{ {m_{1} /\mu_{11} } \right.,m_{2} /\mu_{21} , \ldots ,\left. {m_{n} /\mu_{n1} } \right\}} \right.} \\ {e_{2} = \left\{ {m_{1} /\mu_{12} } \right.,m_{2} /\mu_{22} , \ldots ,\left. {m_{n} /\mu_{n2} } \right\}} \\ \vdots \\ {\left. {e_{L} = \left\{ {m_{1} /\mu_{1L} } \right.,m_{2} /\mu_{2L} , \ldots ,\left. {m_{n} /\mu_{nL} } \right\}} \right\},} \\ \end{array}$$
(5)

where \(\mu_{ij}\) indicates the degree to which the evaluated MWS \(m_{i}\) matches the synthesized parameter \(e_{j}\)\((j = 1,2, \ldots ,L)\). The value of \(\mu_{ij}\) is based on the following two cases:

If \(C_{1} \cap C_{2} \cap \cdots \cap C_{p} = \emptyset\), that is, the individual evaluation indicators of the evaluation experts are completely different, then:

$$\mu_{ij} = {\text{min}} \left\{ {\text{min} \left\{ {v_{ij}^{k} } \right\}\left| {i \in \left\{ {1, \cdots ,n} \right\},j \in \left\{ {1,2, \cdots ,l_{k} } \right\},k \in \left\{ {1,2, \ldots ,m} \right\}} \right.} \right\}.$$
(6)

If \(C_{1} \cap C_{2} \cap \cdots \cap C_{p} \ne \emptyset\), that is, the individual evaluation indicators of the evaluation experts are the same, then,

$$\begin{aligned} \mu_{ij} & = \hbox{min} \left\{ {\hbox{min} \left\{ {v_{ij}^{k} } \right\},\lambda_{i} \left| {i \in \left\{ {1, \ldots ,n} \right\},j \in \left\{ {1,2, \ldots ,l_{k} } \right\},k \in \left\{ {1,2, \ldots ,m} \right\}} \right.} \right\}, \\ \lambda_{i} & = \text{average}\left\{ {v_{ij}^{k} \left| {k \in \left\{ {k_{1} ,k_{2} , \ldots ,k_{m} } \right\}} \right.} \right\}. \\ \end{aligned}$$
(7)

At present, most of the fuzzy soft set methods only focus on different situations between parameter sets, which may not agree with reality. The method in this paper takes into account the existence of parameter set intersections.

  1. 3.

    Overall evaluation

On the basis of \((G,E)\), we can calculate the weighted comparison matrix C. \(\forall x,y \in \left\{ {1,2, \ldots ,n} \right\}\), \(\forall l \in \left\{ {1,2, \ldots ,L} \right\}\), \(\forall i_{1} ,i_{2} , \ldots ,m \in \left\{ {1,2, \ldots ,n} \right\}\),\(\varvec{C} = \left( {c_{xy} } \right)_{n \times n}\), where

$$c_{xy} = \sum\limits_{j} {\hat{\omega }_{l} \gamma_{xy}^{j} } ,$$
(8)
$$\gamma_{xy}^{j} = \left\{ {\begin{array}{*{20}c} {1,} & {u_{xj} \ge u_{yj} ,} \\ {0,} & {u_{xj} < u_{yj} ,} \\ \end{array} } \right.$$
(9)
$$\hat{\omega }_{l} = \omega_{{i_{1} }}^{1} \omega_{{i_{2} }}^{2} \omega_{{i_{m} }}^{m} ,$$
(10)

and \(c_{xy}\) indicates the number of evaluation indicators that are assigned with higher values for \(m_{x}\) than assigned to them for \(m_{y}\). In general, \(0 \le c_{xy} \le 1\), \(c_{xy} = 1\) if and only if \(x = y\). If \(c_{xy} > c_{yx}\), then \(m_{x}\) is better than \(m_{y}\), and vice versa. \(\hat{\omega }_{l}\) indicates the weight of the synthesized parameter.

According to the weighted comparison matrix C, the dominance score \(s(m_{x} )\), \(s(m_{y} )\) can be calculated by

$$s(m_{x} ) = \sum\limits_{y = 1}^{n} {(c_{xy} - c_{yx} )} ,$$
(11)
$$s(m_{y} ) = \sum\limits_{x = 1}^{n} {(c_{yx} - c_{xy} )} ,$$
(12)

where \(s(m_{x} )\) shows the superiority of \(m_{x}\) in the alternative MWS. The higher the dominance score of \(s(m_{x} )\), the better the corresponding MWS \(m_{x}\).The same is true for \(s(m_{y} )\).

4 Case study

This section presents a case study of our proposed MWS evaluation framework. We take a rail line section from the Yibinnan station to Dengjiawan station on the Neijiang to Kunming railway as an example. This rail section is a single-track rail line operated by the Chengdu Railway Bureau in China. It covers a distance of approximately 185 km and has 19 stations. The planner needs to arrange a 90-min segmented rectangular-shaped MW in the timetable for maintenance work.

With the help of the Railway Train Timetable Generation System developed by the National Railway Train Diagram Research and Training Centre in China, the planners have developed three alternative MWS (see Fig. 3), denoted by \(M = \left\{ {m_{1} ,m_{2} ,m_{3} } \right\}\). The evaluation indicator set is \(C = \left\{ {c_{1} ,c_{2} ,c_{3} ,c_{4} ,c_{5} } \right\}\), where \(c_{1} ,c_{2} ,c_{3} ,c_{4} ,\text{and }c_{5}\), respectively, indicate “The basic indicator of MWS is good”, “The design of MWS is not difficult”, “The influence of MWS on transportation organization is small”, “Coordination of the MWS is good”, and “MWS have a high degree of safety implementation”.

Fig. 3
figure 3

Alternative MWSs for Yibinnan–Dengjiawan railway line. a MWS 1, b MWS 2, c MWS 3

Full size image

Evaluation experts set can be expressed as \(EX = \left\{ {ex_{1} ,ex_{2} ,ex_{3} } \right\}\), which represent three experts from the transportation department, dispatch department, and maintenance department, respectively. Each domain expert scores only the indicators they are familiar with. The expert also needs to determine the relative indicator weights based on personal knowledge and expertise.

The evaluation indicator set of experts from the transportation department is \(E_{1} = \left\{ {c_{1} ,c_{2} ,c_{4} } \right\}\), with indicator weights of \(\omega_{1}^{1} = 0.4\),\(\omega_{2}^{1} = 0.3\),\(\omega_{4}^{1} = 0.3\). The evaluation indicator set of experts from the transportation department is \(E_{2} = \left\{ {c_{1} ,c_{4} ,c_{5} } \right\}\), with indicator weights of \(\omega_{1}^{2} = 0.2\),\(\omega_{4}^{2} = 0.3\),\(\omega_{5}^{2} = 0.5\). The evaluation indicator set of experts from the transportation department is \(E_{3} = \left\{ {c_{1} ,c_{3} ,c_{5} } \right\}\), with indicator weights of \(\omega_{1}^{3} = 0.1\),\(\omega_{3}^{3} = 0.4\),\(\omega_{5}^{3} = 0.5\).

Based on their knowledge and experience, each expert provided corresponding evaluation matrices \(\varvec{V}_{1}\),\(\varvec{V}_{2}\), and \(\varvec{V}_{3}\) as follows:

$$\varvec{V}_{1} = \begin{array}{*{20}c} {\begin{array}{*{20}c} {} \\ {m_{1} } \\ {m_{2} } \\ {m_{3} } \\ \end{array} } & {\begin{array}{*{20}c} {\begin{array}{*{20}c} {c_{1} } & {c_{2} } & {c_{4} } \\ \end{array} } \\ {\left[ {\begin{array}{*{20}c} {0.8} & {0.6} & {0.7} \\ {0.7} & {0.7} & {0.7} \\ {0.8} & {0.5} & {0.8} \\ \end{array} } \right]} \\ \end{array} } \\ \end{array} ,$$
$$\varvec{V}_{2} = \begin{array}{*{20}c} {\begin{array}{*{20}c} {} & { \, c_{1} } & { \, c_{4} } & { \, c_{5} } \\ \end{array} } \\ {\begin{array}{*{20}c} {\begin{array}{*{20}c} {m_{1} } \\ {m_{2} } \\ {m_{3} } \\ \end{array} } & {\left[ {\begin{array}{*{20}c} {0.7} & {0.6} & {0.8} \\ {0.7} & {0.7} & {0.8} \\ {0.6} & {0.7} & {0.6} \\ \end{array} } \right]} \\ \end{array} } \\ \end{array} ,$$
$$\varvec{V}_{3} = \begin{array}{*{20}c} {\begin{array}{*{20}c} {} & { \, c_{1} } & { \, c_{3} } & { \, c_{5} } \\ \end{array} } \\ {\begin{array}{*{20}c} {\begin{array}{*{20}c} {m_{1} } \\ {m_{2} } \\ {m_{3} } \\ \end{array} } & {\left[ {\begin{array}{*{20}c} {0.8} & {0.9} & {0.7} \\ {0.8} & {0.9} & {0.7} \\ {0.8} & {0.7} & {0.5} \\ \end{array} } \right]} \\ \end{array} } \\ \end{array} .$$

Then, according to Eq. (3), we transform the evaluation matrices \(\varvec{V}_{1}\),\(\varvec{V}_{2}\), and \(\varvec{V}_{3}\) into fuzzy soft sets \((F_{1} ,C_{1} )\),\((F_{2} ,C_{2} )\), and \((F_{3} ,C_{3} )\).

$$(F_{1} ,C_{1} ) = \begin{array}{*{20}c} {\left\{ {c_{1} = \left\{ {{{m_{1} } \mathord{\left/ {\vphantom {{m_{1} } {0.8}}} \right. \kern-0pt} {0.8}},{{m_{2} } \mathord{\left/ {\vphantom {{m_{2} } {0.7}}} \right. \kern-0pt} {0.7}},{{m_{3} } \mathord{\left/ {\vphantom {{m_{3} } {0.8}}} \right. \kern-0pt} {0.8}}} \right\}} \right.} \\ {c_{2} = \left\{ {{{m_{1} } \mathord{\left/ {\vphantom {{m_{1} } {0.6}}} \right. \kern-0pt} {0.6}},{{m_{2} } \mathord{\left/ {\vphantom {{m_{2} } {0.7}}} \right. \kern-0pt} {0.7}},{{m_{3} } \mathord{\left/ {\vphantom {{m_{3} } {0.5}}} \right. \kern-0pt} {0.5}}} \right\}} \\ {c_{4} = \left\{ {{{m_{1} } \mathord{\left/ {\vphantom {{m_{1} } {0.7}}} \right. \kern-0pt} {0.7}},{{m_{2} } \mathord{\left/ {\vphantom {{m_{2} } {0.7}}} \right. \kern-0pt} {0.7}},{{m_{3} } \mathord{\left/ {\vphantom {{m_{3} } {\left. {0.8} \right\}}}} \right. \kern-0pt} {\left. {0.8} \right\}}}} \right\},} \\ \end{array}$$
$$(F_{2} ,C_{2} ) = \begin{array}{*{20}c} {\left\{ {c_{1} = \left\{ {{{m_{1} } \mathord{\left/ {\vphantom {{m_{1} } {0.7}}} \right. \kern-0pt} {0.7}},{{m_{2} } \mathord{\left/ {\vphantom {{m_{2} } {0.7}}} \right. \kern-0pt} {0.7}},{{m_{3} } \mathord{\left/ {\vphantom {{m_{3} } {0.6}}} \right. \kern-0pt} {0.6}}} \right\}} \right.} \\ {c_{4} = \left\{ {{{m_{1} } \mathord{\left/ {\vphantom {{m_{1} } {0.6}}} \right. \kern-0pt} {0.6}},{{m_{2} } \mathord{\left/ {\vphantom {{m_{2} } {0.7}}} \right. \kern-0pt} {0.7}},{{m_{3} } \mathord{\left/ {\vphantom {{m_{3} } {0.7}}} \right. \kern-0pt} {0.7}}} \right\}} \\ {\left. {c_{5} = \left\{ {{{m_{1} } \mathord{\left/ {\vphantom {{m_{1} } {0.8}}} \right. \kern-0pt} {0.8}},{{m_{2} } \mathord{\left/ {\vphantom {{m_{2} } {0.8}}} \right. \kern-0pt} {0.8}},{{m_{3} } \mathord{\left/ {\vphantom {{m_{3} } {0.6}}} \right. \kern-0pt} {0.6}}} \right\}} \right\},} \\ \end{array}$$
$$(F_{3} ,C_{3} ) = \begin{array}{*{20}c} {\left\{ {c_{1} = \left\{ {{{m_{1} } \mathord{\left/ {\vphantom {{m_{1} } {0.8}}} \right. \kern-0pt} {0.8}},{{m_{2} } \mathord{\left/ {\vphantom {{m_{2} } {0.8}}} \right. \kern-0pt} {0.8}},{{m_{3} } \mathord{\left/ {\vphantom {{m_{3} } {0.8}}} \right. \kern-0pt} {0.8}}} \right\}} \right.} \\ {c_{3} = \left\{ {{{m_{1} } \mathord{\left/ {\vphantom {{m_{1} } {0.9}}} \right. \kern-0pt} {0.9}},{{m_{2} } \mathord{\left/ {\vphantom {{m_{2} } {0.9}}} \right. \kern-0pt} {0.9}},{{m_{3} } \mathord{\left/ {\vphantom {{m_{3} } {0.7}}} \right. \kern-0pt} {0.7}}} \right\}} \\ {\left. {c_{5} = \left\{ {{{m_{1} } \mathord{\left/ {\vphantom {{m_{1} } {0.7}}} \right. \kern-0pt} {0.7}},{{m_{2} } \mathord{\left/ {\vphantom {{m_{2} } {0.7}}} \right. \kern-0pt} {0.7}},{{m_{3} } \mathord{\left/ {\vphantom {{m_{3} } {0.5}}} \right. \kern-0pt} {0.5}}} \right\}} \right\}.} \\ \end{array}$$

Performing “AND” operations on the fuzzy soft sets \(\left( {F_{1} ,C_{1} } \right)\), \((F_{2} ,C_{2} )\) and \((F_{3} ,C_{3} )\). Then \(\left( {G,E} \right) = (G,C_{1} \times C_{2} \times C_{3} ) = F_{1} ,C_{1} ) \wedge \left( {F_{2} \, ,C_{2} } \right) \wedge \left( {F_{3} \, ,C_{3} } \right)\). As \(C_{1} \cap C_{2} \cap C_{3} \ne \emptyset\), the number of synthetic parameters in fuzzy soft set \(\left( {G,E} \right) = (G,C_{1} \times C_{2} \times C_{3} )\) is less than \(L = 3 * 3 * 3 = 27\).

Assuming that the sets C1, C2, and C3 each provide a parameter to form \(\hat{E} = \left\{ {\hat{e}_{1} ,} \right.\hat{e}_{2} , \ldots ,\left. {\hat{e}_{27} } \right\}\), the parameters of \(\hat{E}\) are as given in Table 5.

Table 5 Parameters of \(\hat{E}\)
Full size table

Table 5 shows that \(\hat{e}_{3} = \hat{e}_{7} = \hat{e}_{9} = c_{1} c_{5}\), \(\hat{e}_{4} = \hat{e}_{19} = \hat{e}_{22} = c_{1} c_{4}\), \(\hat{e}_{5} = \hat{e}_{20} = c_{1} c_{3} c_{4}\), \(\hat{e}_{6} = \hat{e}_{21} = \hat{e}_{25} = c_{1} c_{4} c_{5}\), \(\hat{e}_{12} = \hat{e}_{16} = c_{1} c_{2} c_{5}\), and \(\hat{e}_{24} = \hat{e}_{27} = c_{4} c_{5}\). So the number of parameters in set \(E\) is 18. Let \(E = \left\{ {e_{1} ,e_{2} , \ldots ,e_{18} } \right\}\). The parameters of \(E\) are given in Table 6.

Table 6 Parameters of \(E\)
Full size table

Computing fuzzy soft set \((G,E)\) according to Eq. (5) produces the results given in Table 7. Combining Eq. (8) to (10) and calculating the weight values of the synthesized parameter lead to the results given in Table 8. The weight value indicates the degree of importance of each parameter after synthesis.

Table 7 Fuzzy soft set (G, E)
Full size table
Table 8 The weight values for the synthesized parameter
Full size table

Based on Eq. (8) to (10), the weighted comparison matrix C can be obtained:

$$\varvec{C} = \begin{array}{*{20}c} {\begin{array}{*{20}c} {} \\ {m_{1} } \\ {m_{2} } \\ {m_{3} } \\ \end{array} } & {\begin{array}{*{20}c} {\begin{array}{*{20}c} {m_{1} } & {m_{2} } & {m_{3} } \\ \end{array} } \\ {\left[ {\begin{array}{*{20}c} {1.000} & {0.411} & {0.904} \\ {0.936} & {1.000} & {0.988} \\ {0.128} & {0.156} & {1.000} \\ \end{array} } \right]} \\ \end{array} } \\ \end{array} .$$

Combining Eq. (11) to (12), we can obtain the dominance score of each MWS:

$$s(m_{1} ) = \sum\limits_{k = 1}^{3} {(c_{1k} - c_{k1} ) = 0 + (0.411 - 0.936) + (0.904 - 0.128) = } 0.251,$$
$$s(m_{2} ) = \sum\limits_{k = 1}^{3} {(c_{2k} - c_{k2} ) = (0.936 - 0.411) + 0 + (0.988 - 0.156) = } 1.357,$$
$$s(m_{3} ) = \sum\limits_{k = 1}^{3} {(c_{3k} - c_{k3} ) = (0.128 - 0.904) + (0.156 - 0.988) + 0 = } - 1.608.$$

It can be seen that \(s(m_{2} ) > s(m_{1} ) > s(m_{3} )\), so we can say that \(m_{2}\) is the best one of the three alternative MWSs; \(m_{3}\) is the worst of the three alternatives; \(m_{1}\) is intermediate between \(m_{2}\) and \(m_{3}\).

To further illustrate the effectiveness of fuzzy soft set theory in MWS evaluation, we compare the proposed method with the fuzzy comprehensive evaluation method adopted in Ref. [8]. Assume that the appraisal set of the MWS is V = [very reasonable, relatively reasonable, general, and unreasonable], according to the fuzzy comprehensive evaluation solution steps, we can get the fuzzy evaluation vectors for the three MWSs. The detailed solution process is given in Appendix 2.

$$B^{(1)} = (0.32,0.47,0.21,0),$$
$$B^{(2)} = (0.3,0.57,0.13,0),$$
$$B^{(3)} = (0.22,0.33,0.4,0.05).$$

Based on the principle of maximum membership, we can take the largest proportion of the rating of the appraisals as the overall appraisal result. That is to say, most experts think \(m_{1}\) is relatively reasonable, accounting for 47%. Thirty-two percentage experts think that \(m_{1}\) is very reasonable, and 21% think \(m_{1}\) is general reasonable. For \(m_{2}\) and \(m_{3}\), we can get the similar conclusion.

Considering the fuzzy comprehensive evaluation method cannot directly compare different MWSs, we need to normalize the comprehensive evaluation vector and then rank the three alternative MWSs. The results are as follows:

$$S(m_{2} ) \succ S(m_{1} ) \succ S(m_{3} ).$$

It can be seen that the results obtained by the two methods are consistent. However, the fuzzy comprehensive evaluation method based on the principle of maximum membership can only obtain the largest proportion of the appraisal grades, which will waste part of the evaluation information. In comparison, the proposed fuzzy soft set method allows experts to use individual, personalized sets of evaluation indicators, thus allowing overlap among indicator sets. This is more consistent with reality and can improve evaluation accuracy.

5 Conclusions

MWS evaluation has a feedback on MWS setting, which is an important way to improve the quality of MWS setting. However, there is little research on MWS evaluation. This paper presents an MWS evaluation index system and develops an MWS evaluation framework based on fuzzy soft set theory. The established index system can take into account the needs of different departments. To the best of the authors’ knowledge, this paper is the first attempt to construct an MWS evaluation index system using this approach. Application of fuzzy soft sets to MWS evaluation can effectively describe the experts’ subjective judgment and allow each expert to utilize a set of indicators based on personal preference. In addition, the proposed method integrates information obtained via expert evaluation to obtain an overall evaluation. A case study shows the applicability of this framework to provide a valuable tool for planners to select the best of multiple-alternative MWS.

Future research will focus on improving the MWS evaluation indicator system, as well as investigating other group decision-making methods such as the modified FAHP method [16].