Mini-Batch Adaptive Random Search Method for the Parametric Identification of Dynamic Systems

Abstract

A possible method for estimating the unknown parameters of dynamic models described by differential-algebraic equations is considered. The parameters are estimated using the observations of a mathematical model. The parameter values are found by minimizing a criterion written as the sum of the squared deviations of the values of the state vector’s coordinates from their exact counterparts obtained through measurements at different time instants. Parallelepiped-type constraints are imposed on the parameter values. For solving the optimization problem, a mini-batch method of adaptive random search is proposed, which further develops the ideas of optimization methods used in machine learning. This method is applied for solving three model problems, and the results are compared with those obtained by gradient optimization methods of machine learning procedures and also with those obtained by metaheuristic algorithms.

Appendix

A. Stochastic Gradient Descent (SGD):

$${\theta }^{k+1}={\theta }^{k}-{\alpha }_{k}{\nabla }_{\theta }\ L({\theta }^{k},\hat{x}({t}_{j}),{t}_{j})={\theta }^{k}-{\alpha }_{k}{\nabla }_{\theta }{\underbrace{\left[\mathop{\sum }\limits_{i = 1}^{n}{({\hat{x}}_{i}({t}_{j})-{x}_{i}(\theta ,{t}_{j}))}^{2}\right]}_{L({\theta }^{k},\hat{x}({t}_{j}),{t}_{j})}},$$

where α_k > 0, k = 0, 1, …,  is the step value; t_j is a random time instant on the set T, selected at each kth iteration over again; ∇_θ denotes the gradient with respect to the parameter vector.

B. Classical Momentum (ClassMom):

$$\begin{array}{cc}{\theta }^{k+1}={\theta }^{k}-{\alpha }_{k}{v}^{k},\\ {v}^{k+1}=\beta {v}^{k}+(1-\beta )\ {\nabla }_{\theta }L({\theta }^{k},\hat{x}({t}_{j}),{t}_{j}),\end{array}$$

where v⁰ = o is a zero column vector and β = 0.9. 

C. Nesterov Accelerated Gradient (NAG) for solving the problem $f({x}^{* })={\rm{min}}_{x\in {{\rm{R}}}^{n}}\ f(x)$:

Step 1. Specify the following parameters: the previous weight update γ, where γ ∈ (0, 1), (e.g., γ = 0.9); the learning rate η; an initial point x⁰ ∈ Rⁿ; v⁰ = o; ε₁ > 0.

Set k = 0.

Step 2. Set k = k + 1, and execute:

$${y}^{k}={x}^{k}-\gamma {v}^{k-1},\quad {g}^{k}=\nabla {f}^{k}({y}^{k}),\quad {v}^{k}=\gamma {v}^{k-1}+\eta {g}^{k}.$$

Step 3. Calculate x^k = x^k−1 − v^k.

Step 4. Check the condition $\left\Vert {x}^{k}-{x}^{k-1}\right\Vert <{\varepsilon }_{1}$.

If this condition is satisfied, then x^* = x^k. Otherwise, return to Step 2.

D. Adaptive Gradient Method (AdaGrad) for solving the problem $f({x}^{* })={\rm{min}}_{x\in {{\rm{R}}}^{n}}\ f(x)$.

Step 1. Specify the following parameters: the previous weight update γ, where γ ∈ (0, 1) (e.g., γ = 0.9); the learning rate η (as a rule η = 0.01); an initial point x⁰ ∈ Rⁿ; the smoothing parameter ε = 10⁻⁶ ÷ 10⁻⁸ (also called the fuzz factor); ε₁ > 0; G⁻¹ = o.

Set k = 0. 

Step 2. Set

$${g}^{k}=\nabla {f}^{k}({x}^{k});\quad {G}^{k}={G}^{k-1}+{g}^{k}\odot {g}^{k},$$

where  ⊙  is the Hadamard product of matrices.

Step 3. Calculate

$${x}^{k+1}={x}^{k}-\eta {g}^{k}\oslash \sqrt{{G}^{k}+\varepsilon },$$

where  ⊘  is the operation of element-wise matrix division.

Step 4. Check the condition $\left\Vert {x}^{k+1}-{x}^{k}\right\Vert <{\varepsilon }_{1}$.

If this condition is satisfied, then x^* = x^k+1. Otherwise set k = k + 1, and return to Step 2.

E. Root Mean Square Propagation (RMSProp) for solving the problem $f({x}^{* })={\rm{min}}_{x\in {{\rm{R}}}^{n}}\ f(x)$.

Step 1. Specify the following parameters: the previous weight update γ, where γ ∈ (0, 1) (e.g., γ = 0.9); an initial point x⁰ ∈ Rⁿ; the smoothing parameter ε = 10⁻⁶ ÷ 10⁻⁸; ε₁ > 0; the step value η (as a rule, η = 0.001); M⁻¹ = o.

Set k = 0. 

Step 2. Set g^k = ∇ f^k(x^k), G^k = g^k ⊙ g^k, and M^k = γM^k−1 + (1 − γ)G^k. 

Step 3. Calculate ${x}^{k+1}={x}^{k}-\eta {g}^{k}\oslash \sqrt{{M}^{k}+\varepsilon }.$

Step 4. Check the condition $\left\Vert {x}^{k+1}-{x}^{k}\right\Vert <{\varepsilon }_{1}.$

If this condition is satisfied, then x^* = x^k+1.  Otherwise set k = k + 1, and return to Step 2.

F. Adaptive Moment Estimation (Adam) for solving the problem M[f(x)] → min, which has the random samples f¹(x), f²(x), …, f^K(x). 

Step 1. Specify the following parameters: the step value α = 0.001; the moment estimation parameters β₁ = 0.9 and β₂ = 0.999; an initial point x⁰ ∈ Rⁿ; the smoothing parameter ε = 10⁻⁸; ε₁ > 0; the initial value m⁰ = o of the first vector of moments M[∇f(x)]; the initial value v⁰ = o of the second vector of moments M[∇f(x) ⊙ ∇f(x)]. 

Set k = 0. 

Step 2. Set k = k + 1, 

$$\begin{array}{ccc}{g}^{k}=\nabla {f}^{k}({x}^{k-1});\quad {m}^{k}={\beta }_{1}{m}^{k-1}+(1-{\beta }_{1}){g}^{k};\\ {G}^{k}={g}^{k}\odot {g}^{k};\quad {v}^{k}={\beta }_{2}{v}^{k-1}+(1-{\beta }_{2}){G}^{k};\\ {\hat{m}}^{k}=\frac{{m}^{k}}{1-{{\beta }_{1}}^{k}};\quad {\hat{v}}^{k}=\frac{{v}^{k}}{1-{{\beta }_{2}}^{k}}.\end{array}$$

Step 3. Calculate ${x}^{k}={x}^{k-1}-\alpha {\hat{m}}^{k}\oslash \sqrt{{\hat{v}}^{k}+\varepsilon }$.

Step 4. Check the condition $\left\Vert {x}^{k+1}-{x}^{k}\right\Vert <{\varepsilon }_{1}$.

If this condition is satisfied, then x^* = x^k.  Otherwise, return to Step 2.

G. Adam Method Modification (Adamax) for solving the problem M[f(x)] → min, where f(x) ∈ C¹. The problem has the random samples f¹(x), f²(x), …, f^K(x).

Step 1. Specify the following parameters: the step value α = 0.002; the moment estimation parameters β₁ = 0.9 and β₂ = 0.999, β₂ ∈ [0, 1); an initial point x⁰ ∈ Rⁿ; the smoothing parameter ε = 10⁻⁸; ε₁ > 0; the initial value m⁰ = o of the first vector of moments M[∇f(x)]; u⁰ = o.

Set k = 0.

Step 2. Set k = k + 1,

$$\begin{array}{l}{g}^{k}=\nabla {f}^{k}({x}^{k-1});\quad {m}^{k}={\beta }_{1}{m}^{k-1}+(1-{\beta }_{1}){g}^{k},\\ {u}^{k}={\rm{max}}\left\{{\beta }_{2}{u}^{k-1},\left|{g}^{k}\right|\right\}\quad (\,\text{max is element-wise}\,).\end{array}$$

Step 3. Calculate ${x}^{k}={x}^{k-1}-\frac{\alpha }{1-{{\beta }_{1}}^{k}}{m}^{k}\oslash {u}^{k}$.

Step 4. Check the condition $\left\Vert {x}^{k+1}-{x}^{k}\right\Vert <{\varepsilon }_{1}$.

If this condition is satisfied, then x^* = x^k.  Otherwise, return to Step 2.

H. Nesterov–accelerated Adaptive Moment Estimation (Nadam).

Step 1. Specify the following parameters: the step value α = 0.002; the moment estimation parameters β₁ = 0.975 and β₂ = 0.999; an initial point x⁰ ∈ Rⁿ; the smoothing parameter ε = 10⁻⁸; the initial value m⁰ = o of the first vector of moments M[ ∇ f(x)]; the initial value v⁰ = o of the second vector of moments M[ ∇ f(x) ⊙ ∇ f(x)].

Set k = 0. 

Step 2. Set k = k + 1,

$$\begin{array}{ccc}{g}^{k}=\nabla {f}^{k}({x}^{k-1});\quad {m}^{k}={\beta }_{1}{m}^{k-1}+(1-{\beta }_{1}){g}^{k};\\ {G}^{k}={g}^{k}\odot {g}^{k};\quad {v}^{k}={\beta }_{2}{v}^{k-1}+(1-{\beta }_{2}){G}^{k};\\ {\hat{m}}^{k}=\frac{{\beta }_{1}{m}^{k}}{1-{{\beta }_{1}}^{k+1}}-\frac{(1-{\beta }_{1}){g}^{k}}{1-{{\beta }_{1}}^{k}};\quad {\hat{v}}^{k}=\frac{{\beta }_{2}{v}^{k}}{1-{{\beta }_{2}}^{k}}.\end{array}$$

Step 3. Calculate ${x}^{k}={x}^{k-1}-\alpha {\hat{m}}^{k}\oslash \sqrt{{\hat{v}}^{k}+\varepsilon }$.

Step 4. Check the condition $\left\Vert {x}^{k+1}-{x}^{k}\right\Vert <{\varepsilon }_{1}$.

If this condition is satisfied, then x^* = x^k.  Otherwise, return to Step 2.

I. Mini-batch Gradient Descent:

$${\theta }^{k+1}={\theta }^{k}-{\alpha }_{k}{\nabla }_{\theta }\ \bar{Q}({\theta }^{k}),\quad {\alpha }_{k}>0,\quad k=0,1,\ldots ,$$

where α_k is the step value (learning rate),

$$\overline{Q}(\theta )=\frac{1}{m}\sum _{i\in {J}_{m}}{\underbrace{\left[\mathop{\sum }\limits_{j = 1}^{n}{\theta }_{j}{f}_{j}({x}_{i})-{y}_{i}\right]}\limits_{L(\theta ,{x}_{i},{y}_{i})}}^{2}=\frac{1}{m}\sum _{i\in {J}_{m}}L(\theta ,{x}_{i},{y}_{i}),$$

where J_m is the set of m numbers of arbitrary components (x_i, y_i) ∈ X^l of a learning sample (e.g., m consecutive elements). To implement one improvement of the parameters, not the entire dataset, but its small part is required (as a rule, from 50 to 256 components in applications).