Design and Analysis of a Cluster-Based Intelligent Hybrid Recommendation System for E-Learning Applications

Abstract

Recently, different recommendation techniques in e-learning have been designed that are helpful to both the learners and the educators in a wide variety of e-learning systems. Customized learning, which requires e-learning systems designed based on educational experience that suit the interests, goals, abilities, and willingness of both the learners and the educators, is required in some situations. In this research, we develop an intelligent recommender using split and conquer strategy-based clustering that can adapt automatically to the requirements, interests, and levels of knowledge of the learners. The recommender analyzes and learns the styles and characteristics of learners automatically. The different styles of learning are processed through the split and conquer strategy-based clustering. The proposed cluster-based linear pattern mining algorithm is applied to extract the functional patterns of the learners. Then, the system provides intelligent recommendations by evaluating the ratings of frequent sequences. Experiments were conducted on different groups of learners and datasets, and the proposed model suggested essential learning activities to learners based on their style of learning, interest classification, and talent features. It was experimentally found that the proposed cluster-based recommender improves the recommendation performance by resulting in more lessons completed when compared to learners present in the no-recommender cluster category. It was found that more than 65% of the learners considered all criteria to evaluate the proposed recommender. The simulation of the proposed recommender showed that for learner size values of <1000, better metric values were produced. When the learner size exceeded 1000, significant differences were obtained in the evaluated metrics. The significant differences were analyzed in terms of a computational structure depending on $\left|L\right|$, the recommendation list size, and the attributes of learners. The learners were also satisfied with the accuracy and speed of the recommender. For the sample dataset considered, a significant difference was observed in the standard deviation σ and mean μ of parameters, in terms of the Recall (List, User) and Ranking Score (User) measures, compared to other methods. The devised method performed well concerning all the considered metrics when compared to other methods. The simulation results signify that this recommender minimized the mean absolute error metric for the different clusters in comparison with some well-known methods.

1. Introduction

Currently, e-learning has replaced conventional learning systems to ensure that better objectives are achieved by all learners [1,2,3,4]. The usage of web enhancements to provide online or offline learning to students at any time is called e-learning, which is considered more effective than conventional learning techniques. Personalized e-learning allows learners to access resources wherever and whenever needed in a useful manner [5,6,7].

The current research in e-learning focuses on the development of recommendation methodologies that are expected to achieve better performance compared to the existing recommendation strategies. Hence, it is important to develop a better recommendation system to provide better service to learners. The recommendation system proposed herein consists of some important subsystems, namely, the Learner Subsystem, Domain Subsystem, Application Subsystem, Adaptation Subsystem, and Session Subsystem. This research presents the design of some strategies needed to provide better recommendations compared to the existing state-of-the-art techniques. The proposed method was executed on an educational dataset with 1000 learners. The experimental results allowed us to conclude that the learners from the simulation cluster could complete a course with reduced computational time and more lessons when compared to the no-recommender cluster. It was also found that the proposed model intelligently recommends learning resources based on the characteristics and styles of learning.

Some research gaps in well-known recommendation systems involve greater differences in the measures of performance and greater computational complexity required, resulting in lower recommendation accuracy. Hence, it is important to develop new recommendation methodologies to offset these problems in well-known strategies in order to find solutions to different real-world issues. To address the above-mentioned drawbacks, in this research we describe the development of an intelligent recommender that can adapt automatically to the requirements, interests, and levels of knowledge of the learners. The recommender automatically analyzes and learns the styles and characteristics of learners. The different styles of learning are processed through split and conquer strategy-based clustering. Then, the system provides intelligent recommendations by evaluating the ratings of frequent sequences.

The proposed intelligent hybrid recommender system applies the following new strategies:

• The datasets are separated into different clusters using the split and conquer strategy;
• Better recommendations are updated by generating recommendations from each cluster;
• A cluster-based linear pattern mining strategy is applied to identify the maximal large cluster sequences;
• The maximal large sequences are enhanced using the linear pattern pruning strategy;
• The styles of learning are identified based on the characteristics of learners; and
• The preferences of learning styles are evaluated in different dimensions.

The definitions and notation applied in this recommender model are presented in Section 2. A literature survey of the existing recommendation methods and the need for new recommendation strategies are discussed in Section 3, and the proposed method is explained in Section 4. The experimental results and an analysis of the proposed method are presented in Section 5, and Section 6 concludes this research.

2. A Literature Survey on Recommendation Systems and the Need for New Recommendation Strategies

Some existing e-learning systems were designed based on the methods of learning, types of information, learner characteristics, and other specific features of operational procedures [8,9,10,11]. Content-based collaborative filtering (CF) and mining procedures were developed [12], as were rule-based customized learning systems with fuzzy theory [13], a mutual filtering procedure with maximum likelihood estimation [14], a mutual filtering recommendation for small datasets [15], a Bayesian model-based learning system [16], and a domain perspective model-based recommendation framework [17].

A semantic recommendation with an ontology-based approach was proposed [18], and a self-organizing map-based e-learning recommendation was developed using artificial neural networks [19]. A customized recommendation model using machine learning and clustering was developed to analyze the learning paths of learners [20]. Customized object-based learning with different styles of learning was analyzed using a clustering algorithm [21], and the learning styles and knowledge levels of learners were evaluated using a content filtering approach [22].

The recently developed recommender systems can be classified into different groups, as outlined below [23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73].

2.1. User Profile Recommendation Systems

Personalized recommendation systems have been developed recently for some domains. The User Profile Oriented Diffusion (UPOD) strategy was developed to analyze learner profiles [74]. This methodology makes specific recommendations based on different operations that are executed in the training stage. The referral stage generates recommendations based on learner profile features.

2.2. Content Recommendation Systems

The convolutional neural network (CNN) method with content-based recommendation was discussed in [75]. This method is used to obtain the concealed factors in different media applications. In this method, the textual information is processed in recommending the required contents.

2.3. Hybrid Recommendation Systems

Some hybrid strategies have been developed for recommending movie-based applications [76]. Content-based filtering was also applied in recommending better movies to users. Personalized learning with a hybrid strategy was developed in [77]. The recommendation system gives personalized recommendations using visualizations.

2.4. Filter-Based Recommendation Systems

Some CF-based recommendations have been implemented for user travel recommendations [78]. This recommendation scheme obtained good performance when compared to other schemes.

2.5. Feature-Based Recommendation Systems

A feature-based recommendation system to solve the ERP System and E-Agribusiness datasets was developed [79], enhanced CF was developed for solving MovieLens applications [80], a group recommender strategy was developed in [81,82], and similarity-based recommender systems for different applications were developed in [83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103].

Critiques of the well-known recommender systems in e-learning [23,24,25,26,27,28,29,30] include the difference in the average absolute error, lower accuracy in the recommendation, and longer computing time required during the recommendation [33,34,35,36,37,38,39,40,41,42,43,44,45]. To address these problems in the existing well-known methods, the recommender proposed herein develops an intelligent recommender that can adapt automatically to the requirements, interests, and levels of knowledge of the learners. The recommender automatically analyzes and learns the styles and characteristics of learners. The different styles of learning are processed through a split and conquer based clustering strategy. Then, the system provides intelligent recommendations by evaluating the ratings of frequent sequences.

3. Notation and Definitions

The following notation and definitions are used in the proposed model [46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,103].

• Learning Items
  The learning items of the resources are defined as the collection of learning objects and are defined as Learning-Items = {Item₁, Item₂, Item₃, …, Item_n}.
• Sequence of Item Sets
  The sequence of item sets defines the ordered sequence: (Item₁, Item₂, Item₃, …, Item_n).
• Subset of a Sequence
  A sequence (x₁, x₂, x₃, …, x_n) is a subset of another sequence (y₁, y₂, y₃, …, y_n) if x₁ ⊆ y_a1, x₂ ⊆ y_a2, x₃ ⊆ y_a3, …, x_n ⊆ y_an where a1, a2, a3, …, a_n are integers such that a1 < a2 < a3 < … < a_n.
• Maximal sequence
  If a sequence x is not a subset of some other sequence, then x is said to be a maximal sequence.
• Linear Pattern
  A linear pattern is defined as a maximal sequence.
• Support(s)
  For a sequence s, Support(s) is defined as the proportion of learners supporting s.
• Rating (L)
  The rating of a learner L is defined in vector form as
  Rating(L) = (r₁, r₂, r₃, …, r_j) where r₁, r₂, r₃, …, r_j define the degree of learners’ knowledge for the specific module used in learning.
• Expected Rating (L)
  If $S_L$ defines a set of frequent sequences, then the expected rating is defined as follows:

  $$\overline{r}=\frac{1}{\left|S_L\right|}{\displaystyle \sum }_{j\in S_L}{r}_j.$$

  (1)
• Similarity (a,b)
  The similarity measurement for two learners a and b is computed as follows:

  $$Similarity\left(a,b\right)=\frac{{{\displaystyle \sum }}_{j\in S_a\cap S_b}(r_{aj}-{\overline{r}}_a)\left(r_{bj}-{\overline{r}}_b\right)}{\sqrt{{{\displaystyle \sum }}_{j\in S_a\cap S_b}{(r_{aj}-{\overline{r}}_a)}^2{{\displaystyle \sum }}_{j\in S_a\cap S_b}{(r_{bj}-{\overline{r}}_b)}^2}}.$$

  (2)
• Prediction (a,j)
  The prediction estimation for a learner a with sequence j is defined as follows:

  $$Prediction\left(a,j\right)={\overline{r}}_a+w_{aj}{\displaystyle \sum }_{a\in L_j}Similarity\left(a,b\right)\left(r_{bj}-{\overline{r}}_b\right).$$

  (3)
• Normalized vector
  The normalized vector $w_{aj}$ is defined as follows:

  $$w_{aj}=1/{\displaystyle \sum }_{a^{\prime }\in L_j}Similarity\left(a,b\right).$$

  (4)
• Recall (List, User)
  Let g(L) be the item count that is associated with the valid target users, $Vali{d}_{Users},$ in the recommendation list L and testing set. Let $I_{User}$ be the total item count related to the valid user User $\in Vali{d}_{Users}$. Then, Recall (List, User) is defined as follows:

  $$Recall\left(List, User\right)=\frac{g\left(L\right)}{I_{User}}.$$

  (5)
• Precision (List, User)
  $Precision \left(List, User\right)$ is given as

  $$Precision\left(List, User\right)=\frac{g\left(L\right)}{\left|L\right|}.$$

  (6)
• Rank (i)
  The metric Rank(i) is represented using the following equation:

  $$Rank\left(i\right)=\frac{\mathrm{The} \mathrm{item} \mathrm{position}\mathrm{in} \mathrm{L}}{\mathrm{The} \mathrm{number} \mathrm{of} \mathrm{items} \mathrm{initially} \mathrm{unknown} \mathrm{to} \mathrm{the} \mathrm{user}}.$$

  (7)
• RankingScore (User)

  $$Ranking Score \left(User\right)={\displaystyle \sum }_{j\in I_{User}}\frac{Rank\left(j\right)}{\left|I_{User}\right|}.$$

  (8)
• Mean Absolute Error (MAE)
  For each of the respective appraisals and evaluations of values of $p_i$ and $q_i$, the MAE is defined as follows:

  $$\mathrm{MAE}=\frac{{{\displaystyle \sum }}_{i=1}^N\left|p_i-q_i\right|)}{\mathrm{N}}.$$

  (9)

4. The General Structure of the Proposed Methodology

An intelligent hybrid recommender is developed in this research that permits both the learners and the educators to use teaching resources effectively. The hybrid intelligent recommender is implemented for all the learning courses and recommends the essential learning resources based on the learner preferences, styles of learning, and characteristics [77].

The proposed intelligent recommender consists of different subsystems: the Domain Subsystem, Learner Subsystem, Application Subsystem, Adaptation Subsystem, and Session Subsystem. The operations of each subsystem are tabulated in

Table 1. Subsystems of the proposed recommender.

Subsystem	Operations
Domain Subsystem	Storing the learning resources and different components
Learner Subsystem	Extracting the complete information and features of the learners
Application Subsystem	Applying the operational rules to identify the requirements of learners
Adaptation Subsystem	Identifying the intelligent recommendations to the learners
Session Subsystem	Controlling all subsystems along with the main system

. The overall recommendation system architecture, which comprises these subsystems, is depicted in Figure 1. The recommendation component architecture of the proposed system is shown in Figure 2.

4.1. Novelty and Advantages of the Proposed Method

The devised recommender applies the following strategies:

• Splitting the datasets into various clusters via the split and conquer strategy;
• Generating recommendations in each cluster and updating the recommendations;
• A cluster-based linear pattern mining strategy;
• Finding the maximal large sequences using a linear pattern pruning strategy;
• Identifying the styles of learning based on the characteristics of learners; and
• Evaluating the preferences of learning styles in different dimensions.

The following are the advantages of the proposed recommender algorithm:

• Reduced deviation in the performance measurements when compared to well-known methods;
• Reduced computational complexity of the recommendation process;
• Increased accuracy in the recommendation list generation;
• Better recommendation generation from each cluster;
• Identification of the learners’ learning styles;
• Analysis of learning style preferences using the Index of Learning Styles strategy; and
• Evaluation of variations in learner preferences across the different dimensions.

4.2. The Proposed Intelligent Hybrid Recommender

The proposed intelligent hybrid recommender model takes the learning resources, Learning-Resources, and the learner list, Learner-List, as its inputs. The model outputs a better recommendation list to the learners. The overall flowchart of the proposed intelligent hybrid recommender is shown in Figure 3. The general structure of the proposed model is depicted in Algorithm 1. Learning-Resources and Learner-List are given as inputs to this model. Learning-Resources is the contents or materials which are divided into different units, each of which consists of finite lessons. Each lesson consists of different topics such as an introduction, overview, applications, tutorials, tests, and exercises. Learner-List represents the faculty and learner information. The faculty prepares different learning components and accesses the appropriate authoring tool. The learners can have different learning characteristics such as requirements, preferences, and methods of learning. The learning style characteristics are identified by evaluating preferences for the learning styles in different dimensions.

Algorithm 1 General Structure of the Proposed Intelligent Hybrid Recommender

Input: Learning-Resources, Learner-List

Output: Identification of learning style characteristics

1:  Filter the contents of the entire Learning-Resources and apply indexing

2:  Divide Learning-Resources into the required number of different modules and lessons

3:  Introduce the different examples, tests, and activities for each of the lessons and modules

4:  Define the components to add new learning resources

5:  Identify the styles of learning based on the characteristics of learners

6:  Evaluate the preferences for learning styles in different dimensions

7:  Process the information

8:  Identify the perception of information

9:  Identify the reception of information

10:   Understand the information based on different learners

11:   Print the learning style characteristics

The learning processes of the learners can be organized by the decisions made by the learners themselves. Depending on the preferences of the learners, they can choose their strategies in generating different learning activities. These learning activities are maintained within the proposed intelligent system. The proposed cluster-based linear pattern mining algorithm is applied to extract the functional patterns of the learners. These patterns can also be used to analyze the complete historical information of the learners, starting from the learning of a specific module and lessons to their successful or unsuccessful completion of the modules and lessons.

The proposed cluster-based linear pattern mining strategy is depicted in Figure 4. The learners are completely clustered and their functional patterns are analyzed as shown in the proposed Algorithm 2. The algorithm starts with the set Learning-Items {Item₁, Item₂, Item₃, …, Item_n} and the sequence of item sets (Item₁, Item₂, Item₃, …, Item_n). The maximal sequence among all the defined sequences of items is identified, and Support(s), the support for sequence s, is evaluated. Then, we obtain the maximal and large sequences of a pattern and perform operations such as sorting, identifying the item sets with maximum size, linear sequence transformation, sequencing operation, and maximal operation. Finally, the maximal large sequences of the cluster are identified so that the functional patterns of the learners can be extracted.

Algorithm 2 Proposed Cluster-Based Linear Pattern Mining

Input: Learning items list, the sequence of items in the cluster

Output: The maximal large sequences of the cluster

1: Define the set Learning-Items = {Item1, Item2, Item3, …, Itemn}

2: Define the sequence of Item sets: (Item1, Item2, Item3, …, Itemn)

3: Identify the maximal sequence among all the defined sequences of item sets

4: Evaluate Support(s), the support for sequence s

5: Obtain the maximal and large sequences of a linear pattern

6: Sort the learner sequences based on the primary key Learner-Id

7: Identify the item sets with maximum size

8: Apply transformation of a learner sequence

9: Determine the expected sequences applying the sequencing operation

10: Apply the maximal operation to reduce data redundancy

11: Identify the maximal large sequences of the cluster

The linear patterns which are the subsets of other linear patterns are examined and pruned to obtain the maximal large sequences; redundant data are thereby reduced [24]. The flowchart of the proposed linear pattern pruning strategy is depicted in Figure 5, and the proposed linear pattern pruning algorithm is shown in Algorithm 3.

Algorithm 3 Proposed Linear Pattern Pruning

Input: Linear patterns

Output: Maximal large sequences

1: Identify the set of sequences

2: For all the identified sequences

3:  For all the subsequences in the identified sequences

4:   If the subsequence is not in the identified sequences, then

5:    Prune the subsequence from the sequence list

6: Print the maximal large sequences

Once the learners finish the sequence of learning resources, the proposed cluster-based evaluation evaluates the learners’ gained knowledge. The flowchart of the cluster-based evaluation is shown in Figure 6. The corresponding algorithm is sketched in Algorithm 4. This algorithm interprets the results based on the learners’ percentages of right answers. The sets of similar learners are also identified.

Algorithm 4 Proposed Cluster-Based Evaluation

Input: Completion of learning resources in the cluster

Output: Evaluation of the learners’ gained knowledge in the cluster

1: Check if the sequence of learning resources is completed by the learner

2: Define the levels of learner rating

3: Interpret the results based on the percentage of right answers within the cluster

4: Identify the set of similar learners, if required

The flowchart of the proposed cluster-based intelligent hybrid recommender is shown in Figure 7, and the algorithm is presented in Algorithm 5. The algorithm splits the input resources into different clusters by applying the divide and conquer strategy and applies the proposed algorithms with the new recommendation and enhanced CF strategies to obtain a better recommendation list for the learners.

Algorithm 5 Proposed Cluster-Based Intelligent Hybrid Recommender

Input: Learning-Resources, Learner-List

Output: Recommendation-List, the better recommendation list for the learners

1: Split Learning-Resources and Learner-List into the required number of clusters

2: For each cluster, apply the following operations:

3:  Apply Algorithm 1 to identify the learning style characteristics of the learners

4:  Apply Algorithms 2 and 3 to identify the maximal large sequences of the cluster

5:  Apply Algorithm 4 to evaluate the learners’ gained knowledge in the cluster

6:  Apply the following recommendation strategy:

7:    Define the rating vector of the learner using Rating(L)

8:    Compare the learners’ ratings

9:    Evaluate the expected rating of the learner using the weighted mean

10:     Determine Similarity(a, b) for learners using Equation (2)

11:     Compute Prediction(a, j) for a learner a with sequence j

12:     Recommend the required activities to the learner based on the evaluation

13: Select the best recommendation from the recommended activities in all clusters evaluation

There are different methods available to explore and analyze the different learning style preferences of learners. The Index of Learning Styles is an objective method used to evaluate the learning style preferences of learners in different dimensions [32,33]. Some of these learning style characteristics are tabulated in

Table 2. Learning style characteristics of learners.

Active	Reflexive
Tasks in the clusters	Tasks outside of the clusters
Willingness to add new resources	Requiring time to think
Experimental	Theoretical
Visual	Verbal
Images, flowcharts	Complete paragraphs
Global	Sequential
Linear understanding	Fast understanding

[32,33].

In the proposed method, the learning style preferences are analyzed using the objective Index of Learning Styles strategy; it evaluates variations in learner preferences across the following different dimensions:

• How to process information for learners;
• Learners’ perceptions of information;
• Learners’ reception of information; and
• Understanding of information based on different learners.

The sequential pattern mining strategy consists of the following different steps which were applied to a database consisting of the columns Learner-Id, Resource Access Time, and Transaction Access Path.

Sorting: The database was sorted using the column Learner-Id. Access to the learning resources was sorted based on the access times. These results are tabulated in

Table 3. Learning resource access based on sorting.

Learner Id	Resource Access Time	Transaction Access Path
1	1 March 2020	Module 1 Lesson 1 Example, Introduction
1	2 March 2020	Module 1 Lesson 2 Example, Introduction
1	3 March 2020	Module 1 Lesson 3 Introduction
2	4 March 2020	Module 1 Lesson 1 Introduction
2	5 March 2020	Module 1 Lesson 1 Example
2	6 March 2020	Module 1 Lesson 2 Introduction
3	7 March 2020	Module 1 Lesson 2 Applications
3	8 March 2020	Module 1 Lesson 3 Exercises
4	9 March 2020	Module 1 Lesson 1 Example
5	10 March 2020	Module 1 Lesson 1 Introduction

.

Finding the large item sets: The large item sets were identified. The outcomes of the mapping of large learning item sets are presented in

Table 4. Mapping of large learning item sets.

Large Learning Item Sets	Mapping
Module 1 Lesson 1 Introduction	A
Module 1 Lesson 1 Overview	B
Module 1 Lesson 1 Applications	C
Module 1 Lesson 1 Flow diagram	D
Module 1 Lesson 1 Limitations	E
Module 1 Lesson 1 Example 1	F
Module 1 Lesson 1 Example 2	G
Module 1 Lesson 1 Example 3	H
Module 1 Lesson 1 Exercises 1	I
Module 1 Lesson 1 Exercises 2	J
Module 1 Lesson 1 Exercises 3	K
Module 1 Lesson 1 Quiz 1	L

.

Transformation: This replaces every transaction in a transformed linear sequence with a set of large item sets. The result of the transformation step is shown in

Table 5. Transformation Function based on Learner-Id.

Learner-Id	Transformation Function
1	<(AB) (CD) (EFGH) (IJ) (KL)
2	<(ABCD) (EFG) (HIJ) (KL)
3	<ABCD) (FGH) (IJKL)
4	<(CDEF) (GH) (IJKL)>
5	<(CDFGH) (IJKL)>

.

Sequencing: In this step, the algorithm applies a set of large item sets to obtain the expected sequences in a fixed number of passes. The large sequences obtained are shown in

Table 6. Large 3, 4, and 5 sequences.

Size	Sequence	Support
3	EFG	1
3	HIJ	1
3	FGH	1
4	EFGH	1
4	ABCD	1
4	IJKL	2
5	CDFGH	1

.

Pruning: The sequential patterns contained within other sequential patterns were pruned to reduce the information redundancy.

Cluster generation based on the learning style characteristics is shown in

Table 7. Cluster generation based on learning style characteristics

Cluster Number	Learning Styles	Number of Learners
1	active, sensing, sequential, global	26
2	reflexive, intuitive, sensing, global	28
3	visual, global, verbal, sensing	42
…	…	…
…	…	…
…	…	…
16	active, sensing, visual, intuitive	57

. Based on the questionnaires, 16 clusters were constructed to determine the profile of the simulation learners’ group. The clusters were constructed for different combinations of learning styles.

The learner profile for the first cluster concerning the first module, which consists of five lessons, is shown in

Table 8. Learner profile for Module 1 and Cluster 1.

Sequence	Learner-Id	Lesson 1	Lesson 2	Lesson 3	Lesson 4	Lesson 5
1	1	2	4	4	4	3
2	2	3	4	4	3	3
3	3	4	4	5	3	4
…	…	…	…	…	…	…
n	1000	5	5	3	2	5

. The provided learner ratings are presented in Table 8. The ratings were measured from a value of 1 for “marginal” to the a value of 5 for “excellent”. The learner profiles were constructed for the remaining modules.

5. Experimental Results and Analysis

In this section, we analyze a simulation of this new recommender executed on some sample datasets. The proposed method was implemented in the Java language. The proposed method was evaluated via different measurements—MAE, Precision (List, User), Recall (List, User), and Ranking Score (User).

5.1. Datasets

The proposed intelligent recommender was simulated on some educational data sets with 1000 learners. The learners were divided into two different clusters: a simulation cluster and a no-recommender cluster. The learners in the no-recommender cluster were not guided through the recommender to access the learning resources. The learners in the simulation cluster were required to go through the proposed model. The simulation cluster consisted of 900 learners and the no-recommender cluster consisted of 100 learners.

The simulation and no-recommender clusters’ data accomplished the normality condition, as indicated by the statistical t-test used to check whether any additional equalization of groups was required. The experimental target was to cluster the simulation cluster learners into subclusters based on the styles of learning and characteristics of the learners. Sixteen clusters were constructed based on the different learning styles and characteristics.

The statistical t-test was conducted to test whether there were any significant differences between the means of the simulation and no-recommender clusters. The learners in these clusters completed the exercises and tests in each chapter and, hence, the intellectual skills of the learners were compared. For the n₁ = 900 learners in the simulation clusters and n₂ = 100 learners in the no-recommender clusters, the averages of the learners’ intellectual skills (simulation cluster: 98.27 and no-recommender cluster: 93.73) were tested at the level of significance α = 5% with n₁ + n₂ − 2 degrees of freedom. The t-test resulted in a calculated value of t_cal = 1.57, less than t_tab = 1.96. This signifies that there were no significant differences between the clusters; hence, it was concluded that additional equalization of the groups was not required.

5.2. Performance Metrics

The performance of the proposed model was analyzed via different measurements. A comparison of the proposed method with the existing CF method in terms of MAE is shown in Figure 8. When the MAE became lower, the recommender predicted learner ratings more effectively. The recommended learning sequences fulfilled the accuracy requirement. The experimental results signify that the proposed method minimized the MAE metric for the different clusters considered when compared to the existing method. A significant difference was inferred in comparison with the CF method.

The proposed recommender’s performance was compared with the existing CF using different metrics as shown in Figure 9. For higher values of Recall (List, User), the system gave better recommendations of the learning resources. For higher values of Precision (List, User), the recommender system signified that there were more items in its recommendation. For lower values of Ranking Score (User), the recommender system signified that the required item was available at the starting place.

5.3. Analysis

The efficiency measure defines the time that learners need to reach the required learning goals. The devised recommender was compared with the Mass Diffusion Heat Spreading Resource (MDHS), nearest neighborhood CF, and User Profile Oriented Diffusion (UPOD) methods. During the execution of the recommender, minimal |L| was set. The experimental results showed that the learners from the simulation cluster could complete a course with reduced computational time and could complete more lessons than those from the no-recommender cluster. It was also observed that the proposed model intelligently recommended learning resources based on the characteristics and styles of learning.

The experimental results of the considered metrics were analyzed and the mean $\mu$ and standard deviation $\sigma$ were calculated for the sample dataset. The simulation outcomes are tabulated in

Table 9. Performance comparison based on $\mathsf{\mu }$.

Strategy	Ranking Score (User)	Recall (List, User)	Precision (List, User)
CF	0.563	0.114	0.005
MDHS	0.282	0.292	0.182
UPOD	0.172	0.301	0.193
Proposed Method	0.070	0.326	0.216

and

Table 10. Performance comparison based on $\sigma$.

Strategy	Ranking Score (User)	Recall (List, User)	Precision (List, User)
CF	0.003	0.004	0.002
MDHS	0.002	0.005	0.002
UPOD	0.002	0.006	0.003
Proposed Method	0.001	0.008	0.004

. The results were compared using the statistical t-test to analyze the significance of the existing methods. From the data in the tables, we concluded that the proposed algorithms outperformed the existing techniques based on the metrics applied to evaluate the performance measurements. For the considered dataset, a significant difference was obtained in terms of parameters $\mu$ and $\sigma$ concerning the Ranking Score (User) and Recall (List, User) measures in comparison with the existing methods. The proposed method also worked well in terms of all metrics in comparison to the existing methods.

The mean computational time of no-recommender versus simulation clusters is plotted in Figure 10. The lessons are plotted on the x-axis and the computational time is measured and plotted on the y-axis. The computational times were measured separately for the no-recommender and simulation clusters. It was experimentally observed that the computational time of simulation cluster learners was significantly reduced.

The expected number of completed lessons for both the no-recommender and simulation clusters is plotted in Figure 11 at different checkpoints considered in the x-axis uniformly. It was experimentally found that the proposed cluster-based recommender improved the performance, as indicated by learners in the simulation cluster completing more lessons than those in the no-recommender cluster category. The number of lessons completed on the y-axis increased over the linear scale as the checkpoint time increased on the x-axis.

The proposed hybrid intelligent recommender was evaluated based on the criteria of simplicity, speed, accuracy, and reliability of the model; this is presented in Figure 12. It was observed that more than 65% of the learners considered all criteria to evaluate the proposed recommender. The learners were also satisfied with the accuracy and speed of the recommender. The proposed strategies were thus effective in obtaining a better recommendation for learners.

6. Conclusions and Future Work

Intelligent recommender systems are required for real-time e-learning applications to enhance performance. A new hybrid intelligent recommender that automatically suits the learning styles and characteristics of the learner was designed in this research. Several simulations were performed on the proposed model, and its performance measurements were compared with those of the existing CF recommender. The proposed sequential pattern clustering, pruning, and recommender algorithms produced better results compared to the existing CF. It was experimentally concluded that the proposed cluster-based recommender improved performance, as indicated by learners in the simulation cluster completing more lessons than those in the no-recommender cluster category. The simulation of the proposed recommender showed that for learner sizes of <1000, better metric values were produced. When the learner size exceeded 1000, significant differences were obtained in the evaluated metrics. The significant differences were analyzed in terms of the computational structure depending on $\left|L\right|$ and learner attributes. It was observed that more than 65% of the learners considered all criteria in evaluating the proposed recommender. The proposed method obtained upper bounds on the Precision and Recall metrics for the sample dataset of 0.326 and 0.216, respectively. The learners were also satisfied with the accuracy and speed of the recommender. The proposed strategies were thus effective in obtaining better recommendations for learners. The experimental results showed that the proposed method minimized the MAE metric for the different clusters considered when compared to the existing method. For the considered sample dataset, a significant difference was observed in the parameters standard deviation $\sigma$ and mean $\mu$ concerning the Recall (List, User) and Ranking Score (User) measures when compared to other methods. The devised method performed well concerning all the considered metrics in comparison to the other methods. It was also found that the proposed model intelligently recommends learning resources based on the characteristics and styles of learning.

Some future research guidelines are as follows [100,101,102,103,104,105]:

• To further enhance and recommend learning resources based on the specific characteristics and learning styles of the learners;
• To apply metaheuristic strategies to further improve the performance metrics;
• To dynamically generate recommendations with minimal complexity; and
• To apply evolutionary operators and machine learning for dynamic and hybrid recommendations.