SIAM Journal on Optimization, Volume 30, Issue 1, Page 439-463, January 2020.
Principal component analysis (PCA) finds the best linear representation of data and is an indispensable tool in many learning and inference tasks. Classically, principal components of a dataset are interpreted as the directions that preserve most of its “energy,” an interpretation that is theoretically underpinned by the celebrated Eckart--Young--Mirsky theorem. This paper introduces many other ways of performing PCA, with various geometric interpretations, and proves that the corresponding family of nonconvex programs has no spurious local optima, while possessing only strict saddle points. These programs therefore loosely behave like convex problems and can be efficiently solved to global optimality, for example, with certain variants of the stochastic gradient descent. Beyond providing new geometric interpretations and enhancing our theoretical understanding of PCA, our findings might pave the way for entirely new approaches to structured dimensionality reduction, such as sparse PCA and nonnegative matrix factorization. More specifically, we study an unconstrained formulation of PCA using determinant optimization that might provide an elegant alternative to the deflating scheme commonly used in sparse PCA.

中文翻译：

---

通过对称函数优化进行的主成分分析没有杂散局部最优

SIAM优化杂志，第30卷，第1期，第439-463页，2020年1月。
主成分分析（PCA）可以找到最佳的数据线性表示形式，并且在许多学习和推理任务中都是必不可少的工具。传统上，数据集的主要成分被解释为保留其大部分“能量”的方向，这一解释理论上受到著名的Eckart-Young-Mirsky定理的支持。本文介绍了执行PCA的许多其他方法，具有各种几何解释，并证明相应的非凸程序族没有伪局部最优，而仅具有严格的鞍点。因此，这些程序的行为类似于凸问题，可以有效地解决全局最优性，例如使用随机梯度下降的某些变体。除了提供新的几何解释并增强我们对PCA的理论理解之外，我们的发现还可能为结构化降维的全新方法铺平道路，例如稀疏PCA和非负矩阵分解。更具体地说，我们使用行列式优化研究了无约束的PCA公式，这可能为稀疏PCA中常用的放气方案提供了一种优雅的替代方案。