Improving classification based on physical surface tension-neural net for the prediction of psychosocial-risk level in public school teachers

Database and data pre-processing

The dataset contains information about the following variables: (i) psychosocial; (ii) physiological, and; (iii) variables associated with pain and musculoskeletal disorders. Psychosocial risk factors may be separated into two main classes: those which have negative effects on health, and those which contribute positively to the worker’s well-being. Although both are present in all work environments, the present study considered those which negatively affect health in academic public-schools organizations (El-Batawi, 1988; Bruhn & Frick, 2011; Lippel & Quinlan, 2011; Weissbrodt & Giauque, 2017; Dediu, Leka & Jain, 2018).

Among the risk factor variables associated with work environment analysis, there was a total of 131 input variables: X_ij = (psychosocial factors, j = 1, …, 123), P_ij = (physiological factors, j =1, … 3) and M_ij = (musculoskeletal symptoms, j = 1, … 5), where, i is the subject under study. Output variables were identified as the level of risk in which the person may be characterized E_ij = Class [low risk (E₁), medium risk (E₂), high risk (E₃), and very high risk (E₄)]. Surface electromyography was performed to corroborate the musculoskeletal problems declared in patients with level of risk medium, high and very high, and confirmed in their clinical history. Electromyography data were collected with a BITalino (2017) (r) evolution Plugged kit (PLUX Wireless Biosignals S.A, Lisbon, Portugal) and validated by a medical specialist to find out if the patients actually had osteomuscular problems.

Redundant psychosocial factors (X_ij) were filtered by means of rank importance of predictors using ReliefF algorithm procedure (1) (Robnik-Š & Kononenko, 2003), with the goal of identifying noisy variables in the dataset using the Chebyshev metric criteria. The ReliefF algorithm located important predictors throughout the 10 nearest neighbors and put the 123 X_ij independent factors into groups. Predictor numbers were listed by ranking, and the algorithm selected the predictors of greatest importance. The weights yielded weight values in the same order as that of the predictors. Distances between factor pairs, at this weight, were measured once again, and the factor with the lowest total value (distance) was chosen, which yielded 12 X_ij factors per group. It further added physiological (P_ij) and musculoskeletal symptom variables (M_ij). The algorithm recognized the variables with the lowest value and punished those predictors (risk associated with each individual X_ir, where r = 1, …, 4 represents the risk level: low risk (X_i1), medium risk (X_i2), high risk (X_i3), and very high risk (X_i4)), which produced different values for neighbors in the same group (risk factors group F_ij), and it increased those which produce different values for neighbors in different groups. ReliefF initially defined predictor weights R_ij at 0, and the algorithm subsequently selected a random value X_ir, iteratively. The k-nearest values X_ir for each group were identified, and all predictor weights F_ij for the nearest neighbors X_iq were updated (Robnik-Šeck & Kononenko, 2003, p. 26). (1)${\displaystyle \mathrm{W}\left[\mathrm{A}\right]:=W\left[A\right]-\sum _{j=1}^k\frac{diff\left(A,R_i,H_j\right)}{m.k}+\sum C\ne class\left(R_i\right)+\left[\frac{P\left(C\right)}{1-P\left(class\left(R_i\right)\right)}\sum _{j=1}^k\frac{diff\left(A,R_i,M_j\left(C\right)\right)}{m.k}\right]}$

Where,

R_i is randomly selected instances.

H_i is k nearest hits (k-nn with the same class). M_j(C) is k nearest misses (k-nn with the different class).

$W\left[A\right]$ is the quality estimation for all attributes A for R_i, and H_j and misses values M_j(C).

$1-P\left(class\left(R_i\right)\right)$ is the sum of probabilities for the misses classes.

m is the processing time repeated.

In total, 20 input variables E_ij = X_ij + P_ij + M_ij were selected (Tables 1–3): twelve variables X_ij = (j = 1, …, 12), which constituted psychosocial variables; three physiological variables P_ij = (j = 1, …, 3), and; five variables associated with musculoskeletal symptoms M_ij = (j = 1, …, 5). This variables were normalized, in accordance with Eq. (2). (2)${\displaystyle E_{normalized}=\frac{\left(E-E_{min}\right)}{E_{max}-E_{min}}}$

Where, E corresponds to the variable to be normalized, E_max is the maximum value of each variable, E_min is the minimum value, and E_normalized is the normalized variable within the −1 to 1 range.

Basis of the surface tension-neural net algorithm (PST-NN)

The approach was based on the theory of liquid surface tension (Macleod, 1923; Jasper, 1972; Tyson & Miller, 1977), given by Eq. (3). Liquid surface tension is defined as the amount of energy necessary to increase surface area of a liquid per unit of area. Surface tension (a manifestation of liquid intermolecular forces) is the force that tangentially acts, per unit of longitude, on the border of a free surface of a liquid in equilibrium, and which tends to contract the said surface (Adamson & Gast, 1967a). The cohesive forces between liquid molecules are responsible for a phenomenon known as surface tension (Fowkes, 1962; Adamson & Gast, 1967b; Tida & Guthrie, 1993; Law, Zhao & Strojnisìtva, 2016; Almeida et al., 2016).

Where, γ is the surface tension that measures the force per unit length (in the model γ is the classification risk level), F is the force required to stop the side from starting to slide, L the length of the movable side, and the reason for the 1/2 is that the film has two surfaces (Macleod, 1923). In this model, the multiplication of the perimeter of an object by the surface tension of a liquid yields the force that a liquid exerts on its surface, on an object, in order to prevent said tension from breaking. As such, if the weight of an object is greater than the force exerted by the liquid on its surface, the object tends to sink.

The theory of surface tension addresses cohesion between molecules in a liquid, and their energetic relationship with the exterior, generally a gas. When submitted to a force that breaks the energetic state of molecular cohesion, the surface of a liquid causes the object producing internal force in the liquid to sink. This proposal sought to emulate the surface tension theory in the psychosocial analysis of risk factors present in work environments and their degrees of risk, from the viewpoint of improving a machine learning model. It used and adapted the said theory to improve risk classification and modify the necessary parameters of a neural network (the number of layers, nodes, weights, and thresholds) to reduce data dimensionality, and increase precision.

Implementation of the PST-NN algorithm

variables E_ij became two physical variables, perimeter and mass, throughout an artificial neural network with four layers. Three of these layers constitute the architecture of a standard neural network, with the difference that, the last level contains a new neural network model based on physical surface tension (Adamson & Gast, 1967b). Eighty neurons were used in layers one and two, due to the fact that substantial changes were not registered using more neurons in these layers . Additionally, just two neurons were used for layer 3, in order to annex the new proposed surface tension layer. The architecture of the artificial neural classification network is shown in Fig. 1. This included the three standard neuron layers, as well as a fourth layer with a novel design.

For the initialization of the neural network parameters, the Nguyen-Widrom algorithm was used (Pavelka & Prochazka, 2004; Andayani et al., 2017), in which random parameters were generated. However, the advantage of this was that the parameters distribute the active neural regions much more uniformly in layers, which improved neural network training, as it presented a lower learning rate error from the beginning.

Layer 1 output calculation: The 20 input variables of a specific individual from the training set, a vector called E, went through an initial layer of 80 neurons. Each neuron had 20 parameters, called weights, which multiplied each input variable of vector E. A parameter called bias b was added to this multiplication. It was associated with each neuron, which results in the partial output of Layer 1. This procedure is described throughout the following equation: (4)${\displaystyle y_k^1=\left(\sum _{i=1}^{20}{E}_i\ast w_{k,i}^1\right)+b_k^{1}fork=1to80}$ (5)${\displaystyle y^1=\left\{y_1^1,y_2^1\dots y_{80}^1\right\}}$

Where E_i is the i variable of the individual chosen from the training set, $w_{k,i}^1$ is the k neuron’s weight in Layer 1, which is multiplied by variable i, $b_k^1$ is neuron k’s bias in Layer 1, which is added to the total, and $y_k^1$ is the result of each k neuron. These 80 results were represented by y¹ vector, and y¹ went through a hyperbolic tangent transfer function, as this is a continuous transfer function, and is recommended for pattern-recognition processes (Harrington, 1993).

Layer 1 output is described in the following equation (6)${\displaystyle Y_k^1=\frac{2}{1+e^{-2{\ast y}_k^1}}-1fork=1to80}$ (7)${\displaystyle Y^1=\left\{Y_1^1,Y_2^1\dots Y_{80}^1\right\}}$

Where, e is the exponential function and Y¹ is the final output for Layer 1 and is composed of 80 outputs, one for each neuron.

Layer 2 output calculation: The 80 outputs from Layer 1, Y¹, become the inputs for Layer 2, which presents the same number of neurons as Layer 1. As such, in accordance with the procedure performed in Layer 1, the following equations are obtained: (8)${\displaystyle {y_k}^2=\left(\sum _{i=1}^{80}{Y_i}^1\ast {w_{k,i}}^2\right)+{b_k}^{2}fork=1to80}$ (9)${\displaystyle y^2=\left\{{y_1}^2,{y_2}^2\dots {y_{80}}^2\right\}}$ Where, $Y_i^1$ is the output of neuron i from Layer 1, $w_{k,i}^2$ is the weight of neuron k, associated with the output of neuron i in Layer 1, $b_k^2$ is neuron k’s bias in Layer 2, and y² includes the 80 responses of each neuron, prior to passing through the transfer function. In order to obtain the final output for Layer 2 (Y²) the hyperbolic transfer function is applied: (10)${\displaystyle Y_k^2=\frac{2}{1+e^{-2\ast y_k^2}}-1fork=1to80}$ (11)${\displaystyle Y^2=\left\{Y_1^2,Y_2^2\dots Y_{80}^2\right\}}$ Layer 3 output calculation: The 80 outputs for Layer 2 were the inputs of Layer 3, which contains two neurons (12)${\displaystyle Y_k^3=\left(\sum _{i=1}^{80}{Y}_i^2\ast w_{k,i}^3\right)+b_k^{3}fork=1to2}$ (13)${\displaystyle Y^3=\left\{Y_1^3,Y_2^3\right\}=\left\{m,Per\right\}}$

Where $Y_i^2$ is the output of neuron i in Layer 2, $w_{k,i}^3$ is the weight of neuron k in Layer 3, which multiplies the output of neuron i in Layer 2, and $Y_k^3$ is the final output of each of the two neurons represented in vector Y³. In the approach of Physical Surface Tension Neural Net (PST-NN), these two output variables were then considered mass (m) and perimeter (Per), respectively, which went into a final layer called the surface tension layer. This was composed of four neurons, one neuron for each risk level. Each of these contributed to a balance of power defined by the following equation: (14)${\displaystyle O_k=1-e^{\frac{F}{2L}}fork=1to4}$

Where, (15)${\displaystyle O_k=1-e^{\frac{-m\ast g}{T_k\ast Per}}fork=1to4}$

(16)${\displaystyle O=\left\{O_1,O_2,O_3,O_4\right\}}$

With: (17)${\displaystyle T_k=\left\{22.1;47.7;72.8;425.41\right\}}$

Where m is the mass that corresponds to the output of the first neuron from Layer 3, g is the value of the gravity constant $9.8\frac{m}{s^2}$) (The multiplication of mass times gravity m∗g yields the weight of an object); Per, the perimeter is the output of the second neuron, from Layer 3; and T_k is the value of the surface tension in neuron k, which were associated to the surface tensions of four liquids: Ethanol (22.1), Ethylene glycol (47.7), Water (72.8), and Mercury (425.41) (Surface tension value (mN/m) at 20 °C) (Jasper, 1972). The four liquids shown above were used, as they are common, relatively well-known, and present different surface tensions. Here, the main idea was the relationship that exists between the four surface tensions and the different weights of objects that can break the surface tension of the liquid. For our model, the surface tension of each liquid was similar to each level of psychosocial risk, where the lowest risk level corresponded to the surface tension of the ethanol, and the very high-risk level was equivalent to the surface tension of the mercury. In this sense, when a person has, according to the psychosocial evaluation, a high-risk level, the parameters in the new surface tension neuron will be equivalent to having traveled the surface tension of ethanol, of ethylene glycol, to finally break the surface tension of the Water. Theoretically, at this point the liquid tension will be broken and the classification of the patient under study will be high risk.

The O_k transfer function was used, owing to its behavior. Note that:

Thus, when the force exerted by the weight was greater than that exercised by the liquid, the surface tension was broken (See Fig. 2). When this occurs, O_k tends to be one, and when it does not, the value of O_k tends to be zero.

The correct outputs for the four types of risk must be as shown below: (20)${\displaystyle \left\{\begin{array}{c}Risk1,O=\left\{1,0,0,0\right\}\hfill \\ Risk2,O=\left\{1,1,0,0\right\}\hfill \\ \frac{Risk3,O=\left\{1,1,1,0\right\}}{Risk4,O=\left\{1,1,1,1\right\}}\hfill \end{array}\right.}$ Risk 4 breaks through all surface tensions, while Risk 1 only breaks through the first surface tension.

Computation of the error backpropagation

The four outputs contained in O were compared to the response E_ij, which the neuron network should have yielded, thus calculating the mean squared error: (21)${\displaystyle errorcm=\sum _{k=1}^4\frac{{\left(O_k-E_k\right)}^2}{2}.}$ The following steps calculated the influence of each parameter on neuron network error, through error backpropagation, throughout partial derivatives. The equation below was derived from O_k:

The derivative of the error, regarding neural network output: (22)${\displaystyle \frac{\partial errorcm}{\partial O_k}=\sum _{k=1}^4\left(O_k-E_k\right)}$

The derivative of the error, regarding layer 3 output: (23)${\displaystyle \frac{\partial errorcm}{\partial Y_1^3}=\frac{\partial errorcm}{\partial m}=\frac{\partial errorcm}{\partial O_k}\ast \frac{\partial O_k}{\partial m}}$ (24)${\displaystyle \frac{\partial O_k}{\partial m}=\frac{g}{T_k\ast Per}{e}^{\frac{-m\ast g}{T_k\ast Per}}}$ (25)${\displaystyle \frac{\partial errorcm}{\partial Y_2^3}=\frac{\partial errorcm}{\partial Per}=\frac{\partial errorcm}{\partial O_k}\ast \frac{\partial O_k}{\partial Per}}$ (26)${\displaystyle \frac{\partial O_k}{\partial Per}=\frac{-m\ast g}{T_k\ast Pe{r}^2}{e}^{\frac{-m\ast g}{T_k\ast Per}}}$ (27)${\displaystyle \frac{\partial errorcm}{\partial Y^3}=\left\{\frac{\partial errorcm}{\partial Y_1^3},\frac{\partial errorcm}{\partial Y_2^3}\right\}}$

Derivative of error, according to layer 3 weights: (28)${\displaystyle \frac{\partial errorcm}{\partial w_{k,i}^3}=\frac{\partial errorcm}{\partial Y_k^3}\ast \frac{\partial Y_k^3}{\partial w_{k,i}^3}fork=1to80,withi=1and2}$ (29)${\displaystyle \frac{\partial Y_k^3}{\partial w_{k,i}^3}=Y_i^2}$

Derivative of error, according to layer 3 bias:

Derivative of error, according to layer 2 output: (31)${\displaystyle \frac{\partial errorcm}{\partial Y_i^2}=\sum _{k=1}^2\frac{\partial errorcm}{\partial Y_k^3}\ast \frac{\partial Y_k^3}{Y_i^2}fori=1to80}$ (32)${\displaystyle \frac{\partial errorcm}{\partial Y_i^2}=\sum _{k=1}^2\frac{\partial errorcm}{\partial Y_k^3}\ast w_{k,i}^{3}fori=1to80}$

Derivative of error, according to layer 2 weights: (33)${\displaystyle \frac{\partial errorcm}{\partial y^2}=\frac{\partial errorcm}{\partial Y^2}\ast \frac{\partial Y^2}{\partial y^2}}$ (34)${\displaystyle \frac{\partial Y^2}{\partial y^2}=1-{\left(Y^2\right)}^2}$ (35)${\displaystyle \frac{\partial errorcm}{w_{k,i}^2}=\frac{\partial errorcm}{\partial Y^2}\ast \frac{\partial Y^2}{\partial y^2}\ast \frac{\partial y^2}{w_{k,i}^2}fork,i=1to80}$

Derivative of error, according to layer 2 bias: (36)${\displaystyle \frac{\partial errorcm}{b_k^2}=\frac{\partial errorcm}{\partial Y^2}\ast \frac{\partial Y^2}{\partial y^2}\ast \frac{\partial y^2}{b_k^2}}$ (37)${\displaystyle \frac{\partial y^2}{b_k^2}=1}$

Derivative of error, according to layer 1 output: (38)${\displaystyle \frac{\partial errorcm}{\partial Y^1}=\frac{\partial errorcm}{\partial Y^2}\ast \frac{\partial Y^2}{\partial y^2}\ast \frac{\partial y^2}{\partial Y^1}}$

Derivative of error, according to layer 1 weights:

(39)${\displaystyle \frac{\partial y^2}{\partial Y^1}=w_{k,i}^{3}fori,k=1to80}$ (40)${\displaystyle \frac{\partial errorcm}{\partial y^1}=\frac{\partial errorcm}{\partial Y^2}\ast \frac{\partial Y^2}{\partial y^2}\ast \frac{\partial y^2}{\partial Y^1}\ast \frac{\partial Y^1}{\partial y^1}}$ (41)${\displaystyle \frac{\partial Y^1}{\partial y^1}=1-{\left(Y^1\right)}^2}$ (42)${\displaystyle \frac{\partial errorcm}{\partial w_{k,i}^1}=\frac{\partial errorcm}{\partial Y^2}\ast \frac{\partial Y^2}{\partial y^2}\ast \frac{\partial y^2}{\partial Y^1}\ast \frac{\partial Y^1}{\partial y^1}\ast \frac{\partial y^1}{\partial w_{k,i}^1}}$ (43)${\displaystyle \frac{\partial y^1}{\partial w_{k,i}^1}=E_{i}fork=1to80;i=1to20}$

Derivative of error, according to layer 1 bias:

(44)${\displaystyle \frac{\partial errorcm}{\partial b^1}=\frac{\partial errorcm}{\partial Y^2}\ast \frac{\partial Y^2}{\partial y^2}\ast \frac{\partial y^2}{\partial Y^1}\ast \frac{\partial Y^1}{\partial y^1}\ast \frac{\partial y^1}{\partial b^1}}$ (45)${\displaystyle \frac{\partial y^1}{\partial b^1}=1}$ (46)${\displaystyle \frac{\partial errorcm}{\partial parameters}=\left\{\begin{array}{c}\hfill \frac{\partial errorcm}{\partial b^1},\frac{\partial errorcm}{\partial w^1},\dots \hfill \\ \hfill \frac{\partial errorcm}{\partial b^2},\frac{\partial errorcm}{\partial w^2},\dots \hfill \\ \hfill \frac{\partial errorcm}{\partial b^3},\frac{\partial errorcm}{\partial w^3}\hfill \end{array}\right\}}$

The new parameters in iteration n+1 were calculated throughout the conjugate gradient method: (47)${\displaystyle parameters\left(n+1\right)=parameters\left(n\right)+\eta \left(n\right)\ast d\left(n\right)}$ Where, (48)${\displaystyle \eta \left(n\right)\ast d\left(n\right)}$ Depends on the (49)${\displaystyle \frac{\partial errorcm}{\partial parameters}values.}$

This procedure was repeated, beginning at step in (4) for the remaining training data, thus completing the first iteration. Later, iterations were performed repeatedly until there was an artificial neural network convergence, according with the following three stop criteria: (a) Minimum performance gradient, the value of this minimum gradient is 10⁻⁶. This tolerance was assigned for adequate neuron network learning; (b) Performance, in order to measure neural network performance, the mean squared error was employed. The value to be achieved is zero, so as to avoid presenting neural output errors; (c) Number of Iterations, the training was stopped if 300 iterations were reached. A high number of iterations was chosen, as ideally, it stopped with error criteria.

The code developed in Matlab V9.4 software can be consulted here: https://codeocean.com/capsule/6532855/tree/v1 (Mosquera, Castrillón Gómez & Parra-Osorio, 2019).

Statistical analysis

The data set was divided into training (80%) and test (20%) groups (train/test split) as published in (Vabalas et al., 2019). For the evaluation of the algorithm the following metrics were used (Rose, 2018): (a) Sensitivity, which provides the probability that, given a positive observation, the neural network will classify it as positive (50); (b) Specificity, which provides the probability that, given a negative observation, the neural network will classify it as negative (51); (c) Accuracy, which gives the total neural network accuracy percentage (52) and, (d) the ROC curve by plotting the sensitivity (true-positive rate) against the false-positive rate (1 − specificity) at various threshold settings. Different authors in other studies as have been used the sensitivity, specificity, and, AUC, for the performance statistics within the independent dataset (Le, Ho & Ou, 2017; Do, Le & Le, 2020; Le et al., 2020).

(50)${\displaystyle Sensitivity=\frac{TP}{TP+FN}}$ (51)${\displaystyle Specificity=\frac{TN}{TN+FP}}$ (52)${\displaystyle Accuracy=\frac{TP+TN}{TP+TN+FP+FN}}$

Where TP, TN, FP and FN denote the number of true positives, true negatives, false positives and false negatives, respectively. In order to analyze the stability of the system in the results obtained, a variance analysis, using (53) was performed, to establish whether there were significant differences in the results. In this analysis, representing the response to the variables, T_i, was the effect caused by nth treatment, and εi, the nth experimental error. The information collected must comply with independence and normality requirements. The variance analysis was performed under a confidence interval of 99.5% (Rodriguez, 2007): (53)${\displaystyle Y_i=\mu +T_i+{\epsilon }_i}$

The efficiency of the PST-NN approach was compared with previous published techniques (Mosquera, Parra-Osorio & Castrillón, 2016; Mosquera, Castrillón & Parra, 2018a; Mosquera, Castrillón & Parra, 2018b; Mosquera, Castrillón & Parra-Osorio, 2019), which were applied over the original data included in the present work. Accuracy was the metric used to make the comparison between PST-NN and Decision Tree J48, Naïve Bayes, Artificial Neural Network, Support Vector Machine Linear, Hill Climbing-Support Vector Machine, K-Nearest Neighbors-Support Vector Machine, Robust Linear Regression, and Logistic Regression Models.

Adjustment of the PST-NN approach

The 20 input variables (psychosocial, physiological, and musculoskeletal symptoms) belonging to the 5443 subjects were used to train and test the Physical Surface Tension Neural Net (PST-NN), according with the level of risk in which the person may be characterized (low, medium, high, and very high risk).

Figure 3 shows the mean squared error that was obtained during the training and testing process of the PST-NN approach, as a function of the iterations number that was used in the adjustment of the neural network parameters. The trend of the blue line, corresponding to the training group, showed how the mean squared error rapidly decreases around the first 100 iterations, reaching a plateau for higher values of the iterations. This plateau indicated that the neural net model has reached the parameters optimization and therefore, any additional increment in the number of iterations not significatively improve the parameters adjustment. Concretely, in in this study and for the next results, 108 iterations were considered in the adjustment of the PST-NN parameters. The curve of the mean squared error corresponding to the testing group (red line) showed a similar behavior to the training group. Indeed, the following results were reported only for the test set.

In relation with the layer that represents the surface tension model in the PST-NN approach (Fig. 1), Figure 4 showed the results of the perimeter and mass outputs for each subject in the test group, according with the risk level. The outputs were plotted in a XY graph, where the mass output corresponds to the X axes and the perimeter to the Y axes. As result, it was possible to see that the points were grouped in specific areas as a function of the risks level. In this sense, the types of risk may additionally be interpreted in physical form. Indeed, the highest risk in the graph corresponded to the red crosses, which present mass values which were relatively larger than the rest, along with relatively smaller perimeters, which cause the surface tension of the four liquids to break. The lowest risk (represented in blue with asterisks) had relatively high perimeters and relatively low masses, which cause them to remain on the surface of certain liquids.

The square root of the mass/perimeter relationship was represented in Fig.5. This transformation of the relationship between mass and perimeter was applied only for improved visualization of the separations between the risk levels. The figure showed that the lowest value of the square root of the mass/perimeter relationship corresponded to the lowest risk level and the highest value to the highest risk level.

Classification performance of the PST-NN approach

The specific confusion matrix for the test set (Table 4) showed the performance of the PST-NN algorithm, as a function of the TP, TN, FP and FN. The number of subjects in each target risk group was 116, 117, 347, and 521 for risk levels 1, 2, 3, and 4, respectively. The number of subjects classified by the algorithm in each risk group was 109 (Risk Level 1), 113 (Risk Level 2), 339 (Risk Level 3), and 540 (Risk Level 4).

Table 5 included the values of sensitivity, specificity, accuracy and AUC for each of the risk levels in the test set. The highest sensitivity value was 97.5% (Risk level 4) and the lowest sensitivity value was 77.8% (Risk level 2), indicating that Risk Level 2 was the most difficult type of risk to classify. On the contrary, the best specificity value was obtained in Risk level 1 (98.2%) and the lowest was in Risk level 3 (96.0%). In relation to the accuracy, Risk Level 2 had the lowest value, indicating that the surface tension neural network would correctly classify an individual, with a probability of the 82.7%, to belong or not to this risk level (it includes true positive and true negative cases). The risk levels with the greatest accuracy values were Risk level 1 followed by Risk level 4, with values of 98.85% and 97.37, respectively. Complementary, Figure 6 showed the receiver operating characteristic curves (ROC curve) for each risk level for the test set. Risk level 4 had the best classification with AUC value of 0.984 (Table 5), while Risk level 2 was the one that presents the most confusion on classification (AUC =0.883).

Finally, the performance of the PST-NN approach was compared in terms of accuracy against the results of linear, probabilistic, and logistic models, previously published (see Table 6). The proposed PST-NN method had the best accuracy value (97.31%), followed by Support Vector Machines (92.86%), Hill-Climbing-Support Vector Machines (92.86%), and Artificial Neural Networks (92.83%). The lowest accuracy values were obtained with the Robust Linear Regression (53.47%), and Logistic Regression (53.65%) techniques. The statistical stability analysis, based on the ANOVA method, showed statistically significant differences between PST-NN and the other techniques, in relation to the accuracy values, with p-value <0.05.