Adapting to PSTs’ Pedagogical Changes in Sustainable Mathematics Education through Flipped E-Learning: Ranking Its Criteria with MCDA/F-DEMATEL

Abstract

Mathematics education for sustainable development is perceived as a core keystone, although its concept is related to its typical issue of ecology and economy in the educational realm. Thus, through current information and communication technology (ICT) impacts, altering pedagogy is highly conflicted in teaching/learning mathematics. This research attempted to classify and investigate criteria for mathematics education with a multi-criteria decision analysis/fuzzy-decision-making trial and evaluation laboratory (MCDA/F-DEMATEL) method in the context of the system of flipped e-learning. In particular, the sustainable development of this research focus is in mathematics education in view of pre-service teachers (PSTs) adapting to their pedagogical changes. With the MCDA/F-DEMATEL, the main criteria and sub-criteria were selected after the consultation of a group of experts as follows: mathematics education, sustainable development, and flipped-e-learning technology criteria. Then, with fifteen sub-criteria, the definitive analyses results were gauged with simple additive weighting (SAW) along with their weight coefficients’ calculation, sensitivity analysis (i–v scenarios), and a professional survey. The results described as the most important criteria for adapting to PSTs pedagogical changes in sustainable mathematical education through a flipped e-learning system were the flipped e-learning technology criteria (scenario i), with an 83% positive perception by professionals’ survey; among its sub-criteria, information technology usage contents (0.43) was the most affected variable, with a 42% very likely perception. Scenario iii, which was the slightly more preferred criteria than mathematics education, got a 78% positive perception from professionals’ survey. Therefore, this proposed methodology could be employed to validate the most important sustainable mathematics with flipped e-learning criteria for adapting to PSTs’ pedagogical changes with corresponding education contexts in more long-term learning.

1. Introduction

The concept of sustainability is related to the educational realm, though it is often connected with the issues of economy and ecology [1,2]. With public awareness increasing, sustainable development in education is gaining growing attention to provide a long-term learning course platform. As the leading agency, the United Nations Educational, Scientific and Cultural Organization (UNESCO), which coordinates the Decade of Education for Sustainable Development (DESD) in the United Nations (UN) [1,3], is trying to engage with the goals, principles, practices, and values of sustainability education, along with the programs and actions by organizations linked to the DESD [3,4]. Additionally, for sustainable development, the UN has implemented an agenda for 2030 in which education is perceived as one of the main sustainable development foundations [1,2,5,6]. Mathematics is commonly acknowledged as an education gatekeeper. Students with a decent mathematics comprehension are more likely to have enhanced outcomes and to finish with enhanced instructive traces than underperforming mathematics students [7]. In building a sustainable mathematics education and a lecture teacher education, it can be comprehended and acknowledged that the content and process of mathematics are significant in teaching specialists. The content and knowledge in mathematical education contain structures, general and specific pedagogical topics, and specialized curricular knowledge [8]. As an academic discipline and the act of teaching, pedagogy is the study of how knowledge and skills are imparted and considers interactions in an educational context [8,9,10]. However, because mathematics education plays an important role in realizing ESD, sustainable mathematics education in a higher education context is still in its initial phases, and it a rare application presence in numerous universities, though the ESD has performed various functions in converting humanities by instructing decision-makers [9,10,11]. In these challenging circumstances, mathematics education over one’s life cycle can generate an opportunity to fill a pedagogical niche for flipped e-learning teaching.

Nearly all institutes in the twenty-first century build the utilization of Information and Communication Technology (ICT) as an instrument of teaching and learning [12,13]. Over the last 60 years, ICT’s utilization with unceasing growth had had strong implications in science, technology, engineering, and mathematics (STEM) teaching and learning [13,14]. In institutes, the underlying assumption of ICT’s utilization is that these technologies can positively impact the outcomes of students [15]. Thus, flipped learning as a pedagogical strategy in postsecondary classrooms involving mathematics classes leads to improvements in student outcomes [16,17]. In particular, the degree to which ICT can be realized depends on how a central processing unit (CPU), an instrument for teaching and learning, can be applied; in other words, this degree depends on how a CPU harms pedagogical exercises and exercises in mathematics education [15,18]. On the basis of innovative information and ICTs, e-learning systems in mathematics education have an interrelationship in effectual long-term sustainable development learning, along with various criteria that adapt the pedagogical changes of pre-service teachers (PSTs) [19,20]. However, it is necessary to complete more research on the efficiency and effectiveness of specific flipped e-learning models, even though they are currently typical e-learning occasions [21]. Some recent research in higher education has described detailed research and analysis of mathematics education for sustainable development through flipped e-learning systems [21,22]. The challenge of integrating ICT into sustainable mathematics education that has been concerned with flipped e-learning systems that can bridge the gap of current education has been shown [23]. In spite of these motivations, together with a deficiency of background and literature, we attempted to classify and examine the criteria and sub-criteria of flipped sustainable mathematics education in the context of e-learning systems that adapt to the pedagogical changes of PSTs.

Selecting and identifying criteria for sustainable mathematics education is a critical matter that is now attaining amplified attention because it is in a rather early phase and has limited applications in higher education [24]. In these circumstances, the multi-criteria decision analysis (MCDA) method, a leading method, has been considered and utilized in numerous foundations and disciplines [24,25]. For abundant conditions, the MCDA can advance various decision-making processes through specific clarifications [25,26,27]. The analytic hierarchy process (AHP) is an effective tool that can be used for defining potential criteria for sustainable mathematics education by helping e-learning systems adapt to PSTs’ pedagogical changes, a process that includes assorted criteria and sub-criteria that must be created with the diverse groups of decision-makers [24,25,26]. To plan an operational arrangement amongst the criteria and their values, the fuzzy-decision-making trial and evaluation laboratory (F-DEMATEL) method must be used [28,29]. The MCDA-F-DEMATEL method has analyzed numerous facets of flipped e-learning systems interconnected with applications to indicate whole qualifications and experiences that correspond to both e-learning and conventional practices [29,30,31]. Its process of analytic networks can be applied to a real network communication in various cases [32]. With the prominent characteristics of this operational method, we can adjust flipped e-learning systems in sustainable mathematics education that adapt to PSTs’ pedagogical changes with various decision-makers for multiple criteria.

This research attempted to classify and examine criteria for mathematics education with the MCDA/F-DEMATEL method in the context of the system of flipped e-learning. In particular, in sustainable mathematics development in education, the focus of this research was on PSTs adapting to their pedagogical changes. With the suggested method, the main portrayed, weighted, judged, and consigned criteria and sub-criteria were found to be mathematics education, sustainable development, and flipped-e-learning technology. The analysis of results with fifteen sub-criteria were measured with simple additive weighing (SAW) in accordance with the weight coefficients’ computation and sensitivity analysis (i–v scenarios) in the context of the MCDA/F-DEMATEL method, as well as a professional survey for the implemented scenarios and sub-scenarios. Here, the research questions that this work sought to solve were:

• Research Question 1: Does a method of MCDA/F-DEMATEL serve to classify and rank criteria and sub-criteria for mathematics education in sustainable development in the context of the teaching of flipped e-learning that adapts to PSTs’ pedagogical changes?
• Research Question 2: Does the selected prioritization of criteria and sub-criteria, as examined through the results of SAW and a sensitivity-analysis for mathematics education in sustainable development in the context of the teaching of flipped e-learning that adapts to PSTs’ pedagogical changes, work in real life contexts?
• Research Question 3: Does the implementation of this professional survey show suitability for mathematics education in sustainable development in the context of the teaching of flipped e-learning that adapts to PSTs’ pedagogical changes?

The next section is that of the “Materials and Methods” and outlines various materials and methods associated with this work. The section “Results and Discussion” corresponds to the various analyses and their results, as well as deep discussion for the asked questions and projected objectives. Finally, the conclusions on the basis of previous sections are explained in the “Conclusions” section.

2. Materials and Methods

The proposed method was applied into an introductory mathematics course, which is an essential subject of the bachelor’s degree in Primary Education, Teacher Training College of the University of Extremadura (Spain). This course has a total of 230 students divided into 3 groups. Then, the MCDA/F-DEMATEL method was applied to the course, along with SAW, a sensitivity analysis, and a professional survey in order to identify and analyze the most important criteria and sub-criteria in sustainable mathematics education through flipped e-learning that adapts to PSTs’ pedagogical changes in a long-term e-learning system. A theoretical framework of the MCDA/F-DEMATEL judgement is shown in Figure 1.

2.1. Instructional Context with Targeting Sample

This research was performed on an introductory mathematical subject named “mathematics and its didactics” that is a compulsory subject of the bachelor’s degree in Primary Education, Teacher Training College of the University of Extremadura (Spain). This course is a compulsory subject that has a total of 230 students divided into 3 groups (84, 75, and 71). This is a core program of the second-year course, and it includes various definitions, themes, and theories for mathematics education. The subject content is organized in six units. In accordance with the teaching assigned curriculum, students need a total of 150 hours to attain the projected competences, which are defined as 6 credits. This course contains the basic features of mathematics teaching and learning, teaching and learning properties, technologies usage, mathematics history, and numerous thoughts on mathematics. Thus, as a main cornerstone, the social and cultural prominence of sustainable development was measured to gauge realistic behaviors in the primary education curriculum.

Table 1. Demographic data of the overall 230 participants in 3 groups.

Items		Group 1	Group 2	Group 3
Total		84	75	71
Male		54%	53%	54%
Female		46%	47%	46%
Age		21	21	21
GPA		6.71	6.83	6.91
Educational Background	Social Sciences	72%	69%	65%
	Science	15%	17%	20%
	Arts	1%	1%	0%
	Technology	1%	2%	5%
	Others	11%	11%	10%

shows the target sample’s demographic information within the chosen course. Here, we can observe the sample distribution as a descriptive analysis. There were a little more male students in all 3 groups (54%, 53%, and 54%), with the participants’ average age being 21 years old. The grade point averages (GPA) were (in a scale of 0–10) 6.71, 6.83, and 6.91, respectively, at the beginning of the semester. On the basis of student’s educational background, almost 80% of students in this course did not have a strong mathematics background. This indicated that they had not taken the subject after the mid- and high-school stage. Based on their education and background information, which were found via preliminary analyses from a continuing project, we found that the prior interaction of these PSTs for mathematics education for sustainable development was extremely low. Consequently, this course, as a backbone material for their future teaching, can provide students the ability to give a lecture on mathematics and sustainable development by adapting to the pedagogical changes of students at the primary education level.

2.2. Criteria and Sub-Criteria Selection Process with MCDA/F-DEMATEL

In this MCDA/F-DEMATEL process, the first step was to choose the proper criteria and sub-criteria, which could have had a direct effect on the potential criteria assessment in sustainable development in mathematics education from the point of view of PSTs adapting to pedagogical changes. Here, after the professionals’ consultation with the real data, the authors decided the criteria and sub-criteria. In particular, the AHP (an MCDA technique) was used to extract the criteria’s and sub-criteria’s relative importance weights and to formulate the evaluation system, as is done in specified problems of decision-making [33,34]. The decision makers’ structured pair-wise comparison method (PCM) depends on the apparent importance of each criterion on the basis of certain prearranged values of scale, 1 to 9 and 1 to 1/9 (see

Table 2. The pair-wise comparison of criteria importance values and their mathematical proportions [35].

More Important Numerical Rate	Description	Less Important Numerical Rate
9	More or less tremendous preference or importance	1/9
8	More or less very to tremendously strong preference or importance	1/8
7	More or less very strong preference or importance	1/7
6	More or less strong to very strong preference or importance	1/6
5	More or less strong preference or importance	1/5
4	More or less adequate to strong preference or importance	1/4
3	More or less adequate preference or importance	1/3
2	More or less equivalent to moderate preference or importance	1/2
1	Equivalent preference or importance	1

) [35]. The following attributes, a_ii = 1 and a_ij = 1/a_ji, in this study were followed in a nine-point scale. In the context of a consistency ratio (CR) matrix, each criterion and sub-criterion weight with the PCM could be designated. In particular, with a thumb rule, a CR weight can be coordinated as to be acceptable only in the circumstance that the weight is smaller than 10% [36]. If this is not the circumstance, professionals, as decision-makers, should alter and adapt their weights of decisions.

On the basis of the proposed techniques and methods, three criteria were selected, namely mathematics education, sustainable development, and flipped-e-learning technology. These had fifteen sub-criteria that were combined into the most important criteria and sub-criteria of flipped e-learning systems in sustainable mathematics education that adapt to PSTs’ pedagogical changes. Here, the first mathematics education group included criteria 1–5, the second sustainable development group comprised criteria 6–10, and the third and final flipped-e-learning technology group comprised criteria 11–15. These three different stages of hierarchical organization were utilized for the sustainable mathematics education decision evaluation problem through flipped e-learning systems that adapt to PSTs’ pedagogical changes.

2.2.1. Mathematics Education

The first criteria group comprises the five sub-criteria, specifically university course contents, university course evaluations, university program contents, university professors, and university system updates, all of which are related to the mathematics education evaluation of the selected course that adapts to PSTs’ pedagogical changes, as shown in Figure 1 using the weighted factors from

Table 3. The mathematics education criteria computation of the five sub-criteria, along with a pair-wise comparison matrix.

9-Point Rating Scale: The Mathematics Education
	(1)	(2)	(3)	(4)	(5)
(1) University course contents	1					0.28
(2) University course evaluations	1/4	1				0.09
(3) University program contents	1/3	2	1			0.12
(4) University professors	2	5	4	1		0.45
(5) University system updates	1/4	1/2	1/2	1/6	1	0.06

CI = 0.023, λ_max = 5.09, RI₅ = 1.12, and consistency ratio (CR) = 0.02 < 0.1.

:

• University course contents: University course contents comprise the organization and explanation in mathematics education that adapts to PSTs’ pedagogical changes that are associated with the proposed goals. In this study, the sub-criterion weight was 0.28, as shown in Table 3.
• University course evaluations: University course evaluations comprise the organization and explanation in mathematics education that adapts to PSTs’ pedagogical changes that are associated with the proposed goals. In this study, the sub-criterion weight was 0.09, as shown in Table 3.
• University program contents: University program contents comprise the organization and explanation in mathematics education that adapts to PSTs’ pedagogical changes that are associated with the proposed goals. In this study, the sub-criterion weight was 0.12, as shown in Table 3.
• University professors: University professors comprise the organization and explanation in mathematics education that adapts to PSTs’ pedagogical changes that are associated with the proposed goals. In this study, the sub-criterion weight was 0.45, as shown in Table 3.
• University system updates: University system updates comprise the organization and explanation in mathematics education that adapts to PSTs’ pedagogical changes that are associated with the proposed goals. In this study, the sub-criterion weight was 0.06, as shown in Table 3.

2.2.2. Sustainable Development

The second criteria group comprises the five sub-criteria, specifically physical contents, economic contents, social contents, environmental contents, and morphological contents, all of which are related to the sustainable development evaluation of the selected course and which adapt to PSTs’ pedagogical changes, as shown in Figure 1 using the weighted factors from

Table 4. The sustainable development criteria computation of the five sub-criteria, along with a pair-wise comparison matrix.

9-Point Rating Scale: The Sustainable Development
	(1)	(2)	(3)	(4)	(5)
(1) Physical contents	1					0.28
(2) Economic contents	1/5	1				0.05
(3) Social contents	1/4	2	1			0.09
(4) Environmental contents	3	5	4	1		0.43
(5) Morphological contents	1/3	3	2	1/3	1	0.15

CI = 0.045, λ_max = 5.18, RI₅ = 1.12, and CR = 0.04 < 0.1.

:

• Physical contents: Physical contents comprise the organization and explanation in sustainable development that adapt to PSTs’ pedagogical changes that are associated with the proposed goals. In this study, the sub-criterion weight was 0.28, as shown in Table 4.
• Economic contents: Economic contents comprise the organization and explanation in sustainable development that adapt to PSTs’ pedagogical changes that are associated with the proposed goals. In this study, the sub-criterion weight was 0.05, as shown in Table 4.
• Social contents: Social contents comprise the organization and explanation in sustainable development that adapt to PSTs’ pedagogical changes that are associated with the proposed goals. In this study, the sub-criterion weight was 0.09, as shown in Table 4.
• Environmental contents: Environmental contents comprise the organization and explanation in sustainable development that adapt to PSTs’ pedagogical changes that are associated with the proposed goals. In this study, the sub-criterion weight was 0.43, as shown in Table 4.
• Morphological contents: Morphological contents comprise the organization and explanation in sustainable development that adapt to PSTs’ pedagogical changes that are associated with the proposed goals. In this study, the sub-criterion weight was 0.15, as shown in Table 4.

2.2.3. Flipped E-Learning Technology

The third and final criteria group comprises the five sub-criteria, specifically user interface contents, interactivity, help contents, distinct application contents, application distribution contents, and technology usage contents, all of which are related to the flipped e-learning technology evaluation of the selected course that adapts to PSTs’ pedagogical changes, as shown in Figure 1 using the weighted factors from

Table 5. The flipped e-learning technology criteria computation of the five sub-criteria, along with a pair-wise comparison matrix.

9-Point Rating Scale: The Flipped E-Learning Technology
	(1)	(2)	(3)	(4)	(5)
(1) User interface contents	1					0.17
(2) Interactivity and help contents	1/3	1				0.07
(3) Distinct program contents	1/4	1/2	1			0.05
(4) Application distribution contents	3	4	5	1		0.28
(5) Technology usage contents	4	5	6	2	1	0.43

CI = 0.048, λ_max = 5.19, RI₅ = 1.12, and CR = 0.04 < 0.1.

:

• User interface contents: User interface contents comprise the organization and explanation in flipped e-learning technology that adapt to PSTs’ pedagogical changes that are associated with the proposed goals. In this study, the sub-criterion weight was 0.17, as shown in Table 5.
• Interactivity and help contents: Interactivity and help contents comprise the organization and explanation in flipped e-learning technology that adapt to PSTs’ pedagogical changes that are associated with the proposed goals. In this study, the sub-criterion weight was 0.07, as shown in Table 5.
• Distinct program contents: Distinct program contents comprise the organization and explanation in flipped e-learning technology that adapt to PSTs’ pedagogical changes that are associated with the proposed goals. In this study, the sub-criterion weight was 0.05, as shown in Table 5.
• Application distribution contents: Application distribution contents comprise the organization and explanation in flipped e-learning technology that adapt to PSTs’ pedagogical changes that are associated with the proposed goals. In this study, the sub-criterion weight was 0.28, as shown in Table 5.
• Technology usage contents: Technology usage contents comprise the organization and explanation in flipped e-learning technology that adapt to PSTs’ pedagogical changes that are associated with the proposed goals. In this study, the sub-criterion weight was 0.43, as shown in Table 5.

2.2.4. MCDA/F-DEMATEL

In the MCDA/F-DEMATEL process, the second step is the operational methodology, the F-DEMATEL technique, which is an out-ranking method that computes the coefficients of the extensive criteria and sub-criteria. Fuzzy logic is a robust device that deals with decision-making’s ambiguity, uncertainty, and vagueness and real-world decision-making problems that are made imprecise due to purposes, constraints, and probable movements that are not recognized (see Figure 2) [37]. Here, it is better to modify the linguistic language and weight turning of the fuzzy digits instead of combining assorted thoughts, judgements, motivations, and experiences for individual and/or group decision-makers.

Accordingly, throughout the formula of decision-making, creating fuzzy digits in preparation is a prerequisite. The F-DEMATEL process has the important benefit of revealing the fuzziness setting that directs the flexible fuzziness circumstances [38,39]. Additionally, the F-DEMATEL method has the possibility to employ processes that solve difficulties and problems that arise in individual and/or group decision-making in fuzzy conditions. A fuzzified Likert scale was commissioned in this work to create an average criteria matrix, as described in Equation (1):

$$P= \left[P_{ij}\right]C_i\times C_i$$

(1)

With a direct relationship, an average criteria matrix can be created. Here, the authors and decision-makers contributed to and built a conventional criteria relationship medium after consulting professionals and extensive literature reviews. The combination of the decision-makers’ prioritization value was concluded such that $p_{ij}^e$, where e is a variable that is considered the decision-maker’s predisposition and k is considered the complete weight of the decision-maker, as shown in Equation (2) [40]:

$$p_{ij}=\left(p_{ij}^{\left(l\right)},p_{ij}^{\left(m\right)},p_{ij}^{\left(u\right)}\right)=\left\{\begin{array}{c}{p}_{ij}^{\left(l\right)}=\min \left(p_{ij}^e\right)\\ p_{ij}^{\left(m\right)}=\sqrt[k]{{\displaystyle {\displaystyle \prod }_{i=1}^k}{p}_{ij}^e}\\ p_{ij}^{\left(u\right)}=\max \left(p_{ij}^e\right)\end{array}\right\}$$

(2)

The equation shows the criteria weight coefficients, which can be consistent with total correlation matrices that comprise the cause and effect relationship diagram (CERD):

$$w_i=\sqrt{{\left(D_i+R_i\right)}^2+{\left(D_i-R_i\right)}^2}$$

(3)

Equation (3) standardizes the criteria value coefficients. They can be determined in order to normalize and control the beginning effect matrices, which are executed with Equation (4):

$$w_i=\frac{w_i}{{\sum }_{j=1}^n{w}_i=1}$$

(4)

Then, the criteria value coefficients can be defuzzified. The total relation matrix, as defuzzified characters, is employed along with the Equation (5):

$$A=\left(a^{\left(l\right)}+4a^{\left(m\right)}+a^{\left(u\right)}\right)·6^{-1}$$

(5)

2.3. Validation Process: SAW, Sensitivity Analysis and Survey Modeling

In the MCDA/F-DEMATEL process, the third step is to use the SAW method to find the most suitable criteria in mathematics education for sustainable development through a flipped e-learning system that adapts to PSTs’ pedagogical changes. The process of SAW is relatively simple operational and rational, but it can be affected when it deals with the MCDA; SAW can also be commissioned in diverse conditions. The F-DEMATEL weight, which is obtained with the criteria gained from the participation of decision-makers, was used to make the sub-criteria combination [41]. Meanwhile, it is possible to exhibit different classes of correlations in the middle of the combined criteria by appropriately indicating the vector of the weighting. In this implementation, with an i-th location, the SAW can be delineated like a raster cubicle per an ordered weighted arrangement, namely w = w₁, w₂, ..., w_n:

$$w_j[0,1]$$

where 1, 2, ..., n and

$${\sum }_{j=1}^n{w}_i=1$$

(6)

Then, one can determine the suitability rate in MCDA problems with regard to the grading degree. This process was utilized for the suitability catalogue here through the use of a combined ranking scale of 0–1, with 0 being designated the least important and 1 being designated the most important (each criterion rate is described in Equation (7)). Here, SI is the suitability degree rate, n is the facets’ digit, w_i is the weight facet of i’s normalized and standardized mark, and x_ij is the weight of i’s criterion rate [42]:

$$S{I}_i={\displaystyle \sum }_{j=1}^n{w}_i{x}_{ij}$$

(7)

In the MCDA/F-DEMATEL process, the fourth step is the application of a sensitivity analysis. This is used to authenticate the constancy of the attained results against prejudice. After input from the decision-makers, a sensitivity analysis was employed for flipped e-learning systems in sustainable mathematics education that adapt to PSTs’ pedagogical changes [41,43]. The results realized by the previous sensitivity analysis were analyzed, but it was essential to augment the sample to a broader resolution that produced outlines, strategies, and scenarios for those decision-makers. In particular, by inserting different criteria and sub-criteria weights, a sensitivity analysis could be performed, as shown in

Table 6. Sensitivity analysis application for criteria weights’ mixture.

Scenario	Mathematics Education	Sustainable Development	Flipped-E-Learning Technology
i. Decision-makers’ precedence	0.21	0.14	0.65
ii. Equal weights to criteria	0.33	0.33	0.33
iii. Mathematise education’s precedence	0.50	0.25	0.25
iv. Sustainable development’s precedence	0.25	0.50	0.25
v. Flipped e-learning technology’s precedence	0.25	0.25	0.50

. This analysis resulted in five different scenarios. Accordingly, it was conceivable to present dissimilar correlation classes amongst the merged criteria by correctly selecting a weighting vector.

In the MCDA/F-DEMATEL process, the fifth and final step is the professional survey that, here, was gathered with forty-one answers in total. All the participants in the survey work as educators, professors, researchers, authorities, etc. The majority of participants deal with practical and educational difficulties daily (over 92%). The survey data were gathered through the email questionnaires. Through survey questions, one can solve the faults and obstacles of positive agents, as well as intensify effectiveness by guiding problems to agents who are ready to determine the implemented scenarios [44]. Thus, using current principles, the survey was arranged in such a way that it took out dissimilar and singular inclinations via a considerable series of survey agents [45]. In particular, the survey had three different parts: background information, the preference of five different scenarios along with their sub-criteria in sensitivity analysis, and additional comments and information about the survey.

3. Results and Discussion

With the proposed MCDA/F-DEMATEL method, the final obtained and proved results were demonstrated along with the SAW and sensitivity analysis. Three criteria and fifteen sub-criteria, used as the indicator-based representation, were devised to substantiate the most significant criteria and sub-criteria of flipped e-learning systems in sustainable mathematics education that adapt to PSTs’ pedagogical changes in more long-term learning programs. Subsequently, the most important criteria were attained with the application of SAW, and were used to measure the probability of five scenarios’ in our sensitivity analysis (i–v), as well as the sub-criteria of each scenario that were gathered from a professional survey. Consequently, the final results indicated the foremost relationships for directing a flipped e-learning system through a sustainable mathematical education that adapts to PSTs’ pedagogical changes.

3.1. The Results of Criteria and Sub-Criteria Selection

Figure 3 shows the schematic model of each criterion and each sub-criterion in accordance with a normalized value that was accredited from the 0–1 indicator of suitability. In Section 2.2.1, Section 2.2.2 and Section 2.2.3, each criterion and sub-criterion were labeled in detail, thus extensively validating the whole judgement procedure. Unquestionably, in this study, the authors, as decision-makers, had complete involvement in the weighting and ranking procedure after a consultation with an expert panel and an extensive literature review. Thus, the chosen criteria were classified into three key groups with regard to the sub-criteria, namely sustainable development mathematics education, pedagogical changes, and flipped-e-learning technology. Fifteen sub-criteria were found to have a relationship with the growth practice: (1) university course contents, (2) university course evaluations, (3) university program contents, (4) university professors, (5) university system updates, (6) physical contents, (7) economic contents, (8) social contents, (9) environmental contents, (10) morphological contents, (11) user interface contents, (12) interactivity and help contents, (13) distinct program contents, (14) application distribution contents, and (15) technology usage contents. The first mathematics education group comprised criteria 1–5. The second sustainable development group comprised criteria 6–10. The third sustainable development group comprised criteria 11–15. Here, these four phases of hierarchical organization were employed for the sustainable mathematical education decision evaluation problem through flipped e-learning systems that adapt to PSTs’ pedagogical changes. The weighs of the intermediate criteria were: 0.21 for mathematics education, 0.14 for sustainable development, and 0.65 for flipped e-learning technology. An analysis of MCDA/F-DEMATEL determines the most significant sub-criteria of the system of flipped e-learning in sustainable mathematics education that adapt to PSTs’ pedagogical changes are technology usage contents, and among them, flipped e-learning technology comprises the most important criteria.

3.2. The Results of Validation Process

Table 6 shows five different scenarios shaped by the sensitivity analysis, along with altered and distinctive values that were employed for the three criteria (see Figure 4). The essential criteria for a flipped e-learning system in sustainable mathematics education that adapts to PSTs’ pedagogical changes were acknowledged within the common ranking scale 0–1, i.e., the higher the value was, the more important its criteria and sub-criteria. The five different scenarios i–v could be designated as: i was the most important criteria based on the authors’ weightings and rankings, which were developed from expert consultation and extensive literature review (0.21, 0.14, and 0.65 for the criteria of mathematics education, sustainable development, and flipped e-learning technology, respectively); ii had an equal weight for all criteria groups (all 0.33); iii was the influence and priority of the mathematics education criteria (0.50, 0.25 and 0.25 for the criteria of mathematics education, sustainable development, and flipped e-learning technology, respectively); iv was the influence and priority of the sustainable development criteria (0.25, 0.50 and 0.25 for the criteria of mathematics education, sustainable development and flipped e-learning technology, respectively); and v was the influence and priority of the flipped e-learning technology criteria (0.25, 0.25, and 0.50 for the criteria of mathematics education, sustainable development, and flipped e-learning technology, respectively). All of this can be seen in Table 6.

The MCDA/F-DEMATEL examination conveyed corresponding results for each stimulus, regardless of the markers’ decided weights. As the most significant criteria in flipped e-learning systems in sustainable mathematics education that adapt to PSTs’ pedagogical changes, implementation scenario i was selected (flipped e-learning technology was 0.65 in the 0–1 index appropriateness scale). Additionally, the results of the sensitivity analysis proved that the archetype and configuration molded by the SAW had tremendous determination and relevance. As shown in

Table 7. Professional survey’s results with the participating sample.

Questionnaires’ Answer		Scenarios
		i	ii	iii	iv	v
0	Blank/do not know	1%	6%	6%	7%	5%
1	Very unlikely	1%	6%	15%	11%	8%
2	Unlikely	4%	13%	20%	15%	11%
3	Not unlikely, not likely	11%	20%	12%	16%	16%
4	Likely	54%	38%	34%	36%	38%
5	Very likely	29%	17%	13%	15%	22%
Valuation in five-point Likert scale		4.01	3.29	2.93	3.08	3.40

and Figure 5 and Figure 6, the divided conceptual advances were assessed by the professional survey with the forty-one decision-makers based on a five-point Likert scale (i.e., ‘very likely’ to ‘very unlikely’) [46]. Consequently, these exertions cleared up what the bigger professional sample reproduced, and the possibilities for the five scenarios were analyzed. Scenario i attained a 29% very likely score (83% optimistic awareness (with most likely and likely scores of 29% and 54%, respectively)), while scenario iii obtained a 13% very likely score (with a 47% positively perception and most likely and likely scores of 13% and 34%, respectively)). In accordance with the weightings and rankings of the criteria and survey analysis, it was found that both examiners preferred scenario i, and scenario iii was the least preferred. In scenario i, in particular, flipped e-learning technology (0.65) was the most affected variable, and it was followed by scenario iii when considering the most important criteria to be mathematics education (see Figure 5). Among the sub-criteria of scenario i, information technology usage contents (0.43) was the most affected variable, with a 42% very likely score (see Figure 6). Additionally, scenarios ii and v were dissimilar, but not significantly. Accordingly, the implemented scenarios redirected the possibilities of the applicable implemented scenarios of flipped e-learning systems in sustainable mathematics education that adapt to PSTs’ pedagogical changes.

Different patterns and preferences of the participating professionals generated by the SAW, sensitivity analysis, and survey analysis outcomes reinforced fifteen possible impacts and three criteria. As such, we found that the results presented an indicator-based model, i.e., criteria and sub-criteria classification for flipped e-learning systems in sustainable mathematics education that adapt to PST’s pedagogical changes. Thus, they showed the application possibility of the five different but efficient implemented scenarios. The results gathered from the applied methods could be exploited to confirm most important sustainable mathematics flipped e-learning criteria that adapt to PSTs’ pedagogical changes with equivalent education surroundings and obtainable data. These results show the plausible weaknesses and complications present in choices and actions for flipped e-learning systems in sustainable mathematics education that adapt to PSTs’ pedagogical changes and that have not yet been suitably exploited. Additionally, these results could be used to determine the decision problems within the proposed methodology.

3.3. Discussion

The obtained results in mathematics education for sustainable development show unique information regarding the significant criteria and sub-criteria assortment of different possible impacts through systems of flipped e-learning that adapts to PSTs’ pedagogical changes. This research created a special decision-support scheme for flipped e-learning systems for sustainable mathematics education, as well as numerous applications with decision-makers’ responses. It fulfilled multi-criteria analyses’ niches for decision-making methods in mathematics e-learning systems based on data gathered from unbiased decision-makers.

Though the sustainability concept is related to the educational realm, sustainable mathematics education is still in an early stage and has scarce applications in a higher education [1,2,9]. In these stressful and puzzling situations, mathematics education, as a life-long educational gatekeeper, can generate alterations of pedagogical changes in flipped e-learning teaching [7]. Along with goals of UNESCO’s DESD and the 2030 Agenda for Sustainable Development in the UN, sustainable development in mathematics education is gaining increasing importance in improving long-term teaching and learning programs [3,4,5,6]. To build a teacher education and sustainable mathematics education, it must be understood that teaching professionals’ mathematical content and processes are significant [8]. Regardless of former endeavors in the teaching of flipped e-learning, the absence/lack of background and literature makes its problematic to classify and scrutinize the criteria and sub-criteria of the systems of flipped e-learning in sustainable mathematics education that adapt to PSTs’ pedagogical changes [22]. Accordingly, these proposed operational approaches are required to regulate the sustainable mathematics education of systems of flipped e-learning, along with numerous decision-makers on various criteria [47]. These proposed skills and methods are scarcely applied in either sustainable mathematics education or systems of flipped e-learning. Hence, there has yet to be research to cope with these features together, thus showing the novel advancements of the proposed work.

Thus, the effect of using ICT in sustainable mathematics education in systems of flipped e-learning teaching is that its positively impact student outcomes [15]. As Treacy [48,49] specified, ICT infrastructure and equipment in mathematics education in all levels of institutions, such as primary schools, secondary schools, colleges, and universities, have been considered crucial concerns. However, the extent and achievement of ICT depend on how an CPU is utilized as a teaching/learning tool, especially how gives affects pedagogical performs and exercises in mathematics education [18]. Thus, the use of flipped education as a pedagogical strategy in postsecondary classrooms that are linked with mathematics classes leads to enhancements in student outcomes [16,17]. In particular, in this work, it is necessary to know the pedagogical changes of PSTs when applying the proposed methodology due to the fact that changes in pedagogy strongly affect the teaching and learning of mathematics. Here, e-learning systems in mathematics education have great interrelationships within effectual long-term sustainable development learning, along with various criteria that adapt to the pedagogical changes of PSTs. [19]. We want to spread the general understanding of ICT by studying the various definitions of students in a mathematical classroom model in flipped e-learning systems that adapts to PSTs’ pedagogical changes. Additionally, together with the proposed operational methodology, this understanding can be a scaffolding for the mentioned objectives of the research. Therefore, these combined aspects represent a novel approach in sustainable mathematics education through flipped e-learning systems that adapt to PSTs’ pedagogical changes.

Due to these motivations and the absence of literature and information, we sought to classify and investigate the criteria and sub-criteria of the systems of flipped e-learning in sustainable mathematics education that adapt to PSTs’ pedagogical changes. Accordingly, to adjust sustainable mathematics education, these operational methods are necessary in systems of flipped e-learning with numerous decision-makers. Therefore, the proposed methodology and the obtained results can be used to validate most significant criteria and sub-criteria of mathematics education for sustainable development through the teaching of flipped e-learning that adapts to PSTs’ pedagogical changes (Research Question 1). The main conclusion of this research is that this method could display the most favorable criteria and sub-criteria in flipped sustainable mathematics e-learning systems that adapt to PSTs’ pedagogical changes regarding long-term learning agendas and could postulate their starting prioritization as well. The SAW method had the highest reliability among the criterion methods (Research Question 2). Likewise, the distinctive configurations and probabilities produced by the SAW and sensitivity analysis outcomes reinforced fifteen probable influences and three criteria. It can be ascertained that criteria’s and sub-criteria’s prioritization can be used as an “indicator-based standard” for the systems of flipped e-learning for sustainable mathematics education that adapts to PSTs’ pedagogical changes; within this standard, the efficient five different scenarios (i–v) that were determined by a professional survey can also be used (Research Question 3).

4. Conclusions

The presented combined methodology can be used to classify and investigate criteria for sustainable mathematics education with the MCDA/F-DEMATEL method via systems of flipped e-learning that adapt to the pedagogical changes of PSTs. This method was applied to an introductory mathematics course that is a compulsory subject of the bachelor’s degree in Primary Education, Teacher Training College of the University of Extremadura (Spain). This course has a total of 230 students divided into three groups of 84, 75, and 71 students. Along with the MCDA/F-DEMATEL method, the authors used decision-makers’ consultation and an extensive literature review to identify three criteria—mathematics education, sustainable development, and flipped e-learning technology—and fifteen sub-criteria—(1) university course contents, (2) university course evaluations, (3) university program contents, (4) university professors, (5) university system updates, (6) physical contents, (7) economic contents, (8) social contents, (9) environmental contents, (10) morphological contents, (11) user interface contents, (12) interactivity and help contents, (13) distinct program contents, (14) application distribution contents, and (15) technology usage contents. Then, with SAW and sensitivity analyses, the most suitable criteria and sub-criteria were selected as the flipped e-learning technology (scenario i: 0.65 out of 0–1 in the used index appropriateness scale) and the technology usage subjects (0.43 out of 0–1 in the used index suitability scale), respectively. Finally, based on a survey of forty-one professionals with a five-point Likert scale, scenario i attained a 29% very likely score (with an 83% affirmative awareness and most likely and likely score of 29% and 54%, respectively)), while scenario iii obtained a 13% very likely score (with 47% positive perception and most likely and likely scores of, 13% and 34%, respectively)). Based on criteria weightings and rankings, as well as a survey analysis, it was revealed that both analyses preferred scenario i, and scenario iii was the least preferred. Therefore, for effective flipped e-learning system implementation, the scenarios were found to be feasible in sustainable mathematics education that adapts to PSTs’ pedagogical changes in a more long-term learning course program.

Consequently, this proposed methodology, along with a contextualized educational topic and a solid framework of results, could be employed to validate the most important sustainable mathematics with flipped e-learning criteria that adapt to PSTs’ pedagogical changes with corresponding education contexts and available, necessary data. Likewise, due it its flexible nature, this methodology can be utilized to explain decision problems. In particular, it could support lectures and/or curriculum designers for long-term learning programs. Here, lecturers would not have to apply the mathematical and operational equations by themselves to find out their own appropriate criteria. They could simple apply these suggestions to their course programs, and/or institute management could use some insights from final reflections into staff management and curriculum design. Additionally, this methodology could connect these proposals to future research questions for other researchers who have interests in the proposed topic. Therefore, we believe this could contribute useful information and improve the interest in the topic.