An Effective Principal Singular Triplets Extracting Neural Network Algorithm

Abstract

In this paper, we propose an effective neural network algorithm to perform singular value decomposition (SVD) of a cross-correlation matrix between two data streams. Different from traditional algorithms, the newly proposed algorithm can extract not only the principal singular vectors but also the corresponding principal singular values. First, a dynamical system is obtained from the gradient flow, which is obtained from optimization of a novel information criterion. Then, based on the dynamical system, a stable neural network algorithm, which can extract the left and right principal singular vectors, is obtained. Moreover, by satisfying orthogonality instead of orthonormality, we are able to extract the normalization scale factor as the corresponding singular value. In this case, the principal singular triplet (principal singular vectors and the corresponding singular value) of the cross-correlation matrix can be extracted by using the proposed algorithm. What’s more, the proposed algorithm can also be used for multiple PSTs extraction on the basis of sequential method. Then, convergence analysis shows that the proposed algorithm converges to the stable equilibrium point with probability 1. Last, experiment results show that the proposed algorithm is fast and stable in convergence, and can also extract multiple PSTs efficiently.

Computational Complexity

Here we discuss the computational complexity of the proposed algorithm in comparison with previous work proposed by Hasan [13], Kaiser [17], Kong [18] and Feng [28].

Clearly, computing the product of two matrices with dimensions $m\times n$ and $n\times l$, respectively, requires a total of $m\times n\times l$ multiplications. In this case, it takes mn multiplications for computing both ${\varvec{A}}(k){\varvec{v}}(k)$ and ${\varvec{A}}^T(k){\varvec{u}}(k)$, and takes 2mn multiplications for computing ${\varvec{u}}^T(k){\varvec{A}}(k){\varvec{v}}(k)$. Hence, the proposed algorithm requires a total of $mn+2m+n$ and $mn+m+2n$ multiplications for updating ${\varvec{u}}(k+1)$ and ${\varvec{v}}(k+1)$, respectively. As a result, the computational complexity of the proposed algorithm is $2mn+3m+3n$. Similarly, the computational complexity of other work can also be obtained. A summary of computational complexities of all mentioned algorithm are listed in Table 1. We find that the proposed algorithm has the lowest computational load. Actually, these are two main reason why the proposed algorithm has lower computational complexity. First, different from coupled algorithms, singular value computing in our algorithm is not needed for singular vectors computing, so singular value computing is not necessary in each step. Second, the proposed algorithm does not contain the term ${\varvec{u}}^T(k){\varvec{A}}(k){\varvec{v}}(k)$ which exists in all other compared algorithms.

Table 1 Computational complexity and running time

Next, the running time of all algorithms are obtained and shown in Table 1. In this simulation, similar to (41), a total number of 100000 Gaussian white sequence ${\varvec{x}}(k)$ and ${\varvec{y}}(k)$ with dimension 30 and 20, respectively, are generated for computing. The running time is obtained from the average of a total of 100 runs in MATLAB R2016b. The computer has a CPU of Core i7-10510U and a RAM of 16 GB.

From Table 1, it is found that Hasan’s algorithm has similar computational complexity but less running time compared with other complicated algorithms. This is due to the fact that Hassan’s algorithm has less number of addition calculations than these algorithms. Moreover, the proposed algorithm costs the least time of all five algorithms. In conclusion, the proposed algorithm has both lowest computational complexity and running time.