Entropy-Randomized Projection

Abstract

We propose a new randomized projection method based on entropy optimization of random projection matrices (the ERP method). The concept of compactness indicator of a data matrix, which is stored in projection matrices, is introduced. An ERP algorithm is formulated in the form of a conditional maximization problem for an entropy functional defined on the probability density functions of the projection matrices. A discrete version of this problem is considered, and conditions are obtained for the existence and uniqueness of its positive solution. Procedures are developed for the implementation of entropy-optimal projection matrices by sampling the probability density functions.

APPENDIX

Proof of Theorem 1. Let us use conditions (5.13), (5.14) to represent Eq. (5.12) in the form

$$ \phi (z) = \frac {\sum \limits ^N_{\alpha =l+1} z^{h_{\alpha }} c_{\alpha }}{\sum \limits ^l_{\alpha =1} z^{h_{\alpha }} c_{\alpha }} = 1.$$

(A.1)

According to (5.5), we have $h_{l+1} - h_1 > 0$ and

$$ \phi (z) = z^{(h_{l+1} - h_1)}\frac {\sum \limits ^N_{\alpha =l+2} z^{h_{\alpha }} c_{\alpha } + c_{l+1}}{\sum \limits ^l_{\alpha =2} z^{h_{\alpha }} c_{\alpha } + c_1}.$$

(A.2)

It follows that

$$ \phi (0) = 0. $$

Now we represent the function $\phi (z)$ in the form

$$ \phi (z) = z^{(h_{N} - h_l)}\frac {\sum \limits ^{N-1}_{\alpha =l+1} z^{h_{\alpha } - h_N} c_{\alpha } + c_{N}}{\sum \limits ^{l-1}_{\alpha =1} z^{h_{\alpha } - h_l} c_{\alpha } + c_l}. $$

(A.3)

It can readily be seen that

$$ \phi (\infty ) = + \infty .$$

Consider the derivative

$$ \phi ^{\prime }(z) = \frac {\sum \limits ^l_{\beta =1}\,\sum \limits ^N_{\alpha =l+1}\,c_{\beta }\, c_{\alpha }(h_{\alpha } - h_{\beta }) z^{(h_{\beta }-h_{\alpha })}}{\left (\sum \limits ^l_{\alpha =1} c_{\alpha }\,z^{h_{\alpha }} \right )^2} > 0.$$

(A.4)

Consequently, the function $\phi (z)$ is monotone increasing, and Eqs. (A.1), (A.2) have a unique positive solution $z^*>0$. $\quad \blacksquare$