Abstract
In this paper, numerical methods for solving fractional differential equations by using a triangle neural network are proposed. The fractional derivative is considered Caputo type. The fractional derivative of the triangle neural network is analyzed first. Then, based on the technique of minimizing the loss function of the neural network, the proposed numerical methods reduce the fractional differential equation into a gradient descent problem or the quadratic optimization problem. By using the gradient descent process or the quadratic optimization process, the numerical solution to the FDEs can be obtained. The efficiency and accuracy of the presented methods are shown by some numerical examples. Numerical tests show that this approach is easy to implement and accurate when applied to many types of FDEs.
1. Introduction
Fractional differential equations (FDEs) have been a hot topic in many scientific fields, such as dynamical system control theory, fluid flow, modelling in rheology, dynamic process of self-similar porous structure, diffusion transport similar to diffusion, electric network, and probability statistics [1–9]. These problems in science and engineering sometimes require us to get the solutions of various fractional differential equations. But as we know, it is difficult to find the exact solutions in most cases. So, we have to use numerical methods to solve fractional differential equations.
In the literature, some numerical methods for solving FDEs have been proposed, such as nonlinear functional analysis methods, including monotone iterative technique [10], topological degree theory [11], and fixed point theorem [12]. In addition, someone proposed the following numerical methods: random walk [13], Adomian decomposition method and variational iteration method [14], homotopy perturbation method [15–17], etc.
In recent years, some scholars try to use the neural network to solve differential equations [18–20]. Lagaris et al. [21] proposed an artificial neural network method for solving initial and boundary value problems. In their work, a trial solution is adopted and written as the sum of two parts. The first part satisfies the initial or boundary conditions and does not contain adjustable parameters while the construction of the second part does not affect the initial and boundary conditions. Then, the neural network is trained to satisfy the differential equation at many selected points. The question for this method is that it is difficult to construct the first part of the trial solution and this method cannot be applied to fractional partial differential equations.
Piscopo et al. [22] also introduced a method to find the numerical solutions of many types of differential equations. The proposed method does not depend on the trial solution and therefore has more flexibility in many cases. It can be used for solving many types of ODE and PDE. The two mentioned neural network techniques motivate us to develop more neural network methods to solve FDEs, but how to get the fractional order of the neural network is a difficult problem.
To overcome this difficulty, in this work, we use a triangle base neural network as basis function to propose an alternative method called triangle neural network methods. This paper is organized as follows. In Section 2, we study the fractional derivative of the triangle base neural network and present the numerical method for solving many types of FDEs. In Section 3, we show the efficiency of the proposed method by some numerical examples. Section 4 is the conclusion.
2. Fractional Derivative of Triangle Neural Network and Numerical Algorithm
2.1. Ordinary Fractional Differential Equation
To solve the following fractional initial value problem (1) and boundary value problem (2), where : where and are real functions, , . and are Caputo fractional derivative operators.
We consider the following triangle base neural network 1 (see Figure 1) to approximate the solution of problems (1) and (2), where are weights for the neural networks and are triangle base functions as the following: where are activation function of neurons in the hidden layer of the above neural network and is an integer and .
Let the weight matrix be and the activation matrix be . The triangle base neural network can be written as
When this neural network is used to be the numerical solution of problem (1), the loss function is
For problem (2), The loss function is where are training points. We have two methods to minimize the loss function to get the corresponding numerical solution. One is the gradient descent algorithm, and another one is the optimization process. For both methods, we need to compute the derivative of the triangle neural network. For this purpose, we have the following theorems.
Theorem 1. For given , , then
Proof. Since
We have
Let , we have
We also have the following.
Theorem 2. Given , , then
Proof. From Theorem 1, we have The second part of this theorem can be verified in the same way. Based on Theorems 1 and 2, we can get the derivative of the triangle base neural network.
In fact, let the solution to problems (1) and (2) be
We can get
Thus, we get the loss function for problem (1):
To carry out the gradient descent process, we have for , and we also have for .
So, we can see that getting the numerical solution of (1) is equivalent to finding s by minimizing the loss function . Usually, we have two methods to do this work. One is the gradient descent method, and another one is adopting the optimization process.
The gradient descent method is as below: where is the step size for the gradient descent. If the function is a linear function of , the initial value problem can also be reduced to
That is, which is a quadratic optimization problem. For fractional boundary value problem (2), the numerical solution can be reduced to the following optimization process:
So, there are two methods to solve this problem. One method is to get the solution through the gradient descent method. Another method is using the optimization technique.
2.2. Fractional Partial Differential Equation
For fractional partial differential equation problem where , and problem where . We use the triangle base neural network (see Figure 2) to approximate the solution of problems (22) and (23). The triangle base neural network can be written as where are weights for the import layer in the neural network, are bias parameters for the hidden layer in the neural network, are weights for the export layer in the neural network, and is the activation function of neurons in the hidden layer.
Based on Theorems 1 and 2, we can get the fractional derivative of the neural network as below:
The loss function for problem (22) is where are training points and is the boundary of problem (22) and
The loss function for problem (23) can be given in the same way.
We use the gradient descent algorithm to train the neural network. In fact, we can train the neural network by layers. First, we train the export layer to get s, then the bias parameters to get s in the hidden layer, and finally, we train the import layer to get s and s.
3. Numerical Experiment
3.1. Numerical Test 1
Consider the following example 1: where . The exact solution to this problem is . We let be and use the optimization method when and gradient descent method when , respectively. The computational results are listed in Tables 1 and 2.
3.2. Numerical Test 2
Consider the following example 2 for boundary value problem: where . The exact solution to this problem is .
We let be and , respectively; the computational error is listed in Figure 3.
We let be and . The computational error is listed in Figure 4.
As we see in example 1, the solution becomes more accurate when is increased. And for the boundary value problem, we use two constraints when we use the optimization process.
3.3. Numerical Test 3
Consider the following example 3: where . The exact solution to this problem is .
We use the gradient descent method to solve this problem, and the computational error is listed in Figure 5. In the training process of the neural network, we set a stopping criteria for the computing process to stop. If this stopping criteria cannot be achieved, the computing will be stopped when times of training is completed.
3.4. Numerical Test 4
Consider the following example 4: where . The exact solution to this problem is .
We use the gradient descent method to solve this problem, and the computational error is listed in Figure 6.
4. Conclusion
The neural network method is a promising approach for solving fractional differential equations. The difficulty for this method is how to calculate the fractional derivatives of the involved neural network. In this paper, we propose numerical methods for solving fractional differential equations including the initial problem, boundary value problem, and partial FDEs by using the triangle base neural network and gradient descending method. All the involved fractional derivatives in this work are considered as Caputo type. We first analyze the fractional derivative of the triangle base neural network. Then, based on the loss function, the proposed numerical methods reduce the fractional differential equation into the gradient descent process or the quadratic optimization problem. By carrying out the gradient descent process or the quadratic optimization process, we can get the numerical solutions. Numerical tests show that this approach is easy to implement and the solution is accurate when applied to many types of FDEs.
Data Availability
All the data supporting this study are available within the article.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Authors’ Contributions
All authors contributed equally and significantly to this paper. All authors read and approved the final manuscript.
Acknowledgments
This research is supported by the National Natural Science Foundation of China (No. 61772295).