We consider the problem of minimizing the l1 norm of a linear map over the sphere, which arises in various machine learning applications such as orthogonal dictionary learning (ODL) and robust subspace recovery (RSR). The problem is numerically challenging due to its nonsmooth objective and nonconvex constraint, and its algorithmic aspects have not been well explored. In this paper, we show how the manifold structure of the sphere can be exploited to design fast algorithms with provable guarantees for tackling this problem. Specifically, our contribution is fourfold. First, we present a manifold proximal point algorithm (ManPPA) for the problem and show that it converges at a global sublinear rate. Furthermore, we show that ManPPA can achieve a local quadratic convergence rate when applied to sharp instances of the problem. Second, we develop a semismooth Newton-based inexact augmented Lagrangian method for computing the search direction in each iteration of ManPPA and show that it has an asymptotic superlinear convergence rate. Third, we propose a stochastic variant of ManPPA called StManPPA, which is well suited for large-scale computation, and establish its sublinear convergence rate. Both ManPPA and StManPPA have provably faster convergence rates than existing subgradient-type methods. Fourth, using ManPPA as a building block, we propose a new heuristic method for solving a matrix analog of the problem, in which the sphere is replaced by the Stiefel manifold. The results from our extensive numerical experiments on the ODL and RSR problems demonstrate the efficiency and efficacy of our proposed methods.

中文翻译：

---

双主成分追踪和正交字典学习的流形近点算法


我们考虑最小化球面上线性映射的 l1 范数的问题，该问题出现在各种机器学习应用中，例如正交字典学习（ODL）和鲁棒子空间恢复（RSR）。由于其非光滑目标和非凸约束，该问题在数值上具有挑战性，并且其算法方面尚未得到很好的探索。在本文中，我们展示了如何利用球体的流形结构来设计快速算法，并为解决该问题提供可证明的保证。具体来说，我们的贡献是四重的。首先，我们针对该问题提出了一种流形近端点算法（ManPPA），并表明它以全局亚线性速率收敛。此外，我们表明，当应用于问题的尖锐实例时，ManPPA 可以实现局部二次收敛率。其次，我们开发了一种基于半光滑牛顿的不精确增广拉格朗日方法来计算 ManPPA 每次迭代中的搜索方向，并表明它具有渐近超线性收敛速度。第三，我们提出了 ManPPA 的随机变体 StManPPA，它非常适合大规模计算，并建立了其亚线性收敛速度。事实证明，ManPPA 和 StManPPA 的收敛速度都比现有的次梯度型方法更快。第四，使用 ManPPA 作为构建块，我们提出了一种新的启发式方法来解决问题的矩阵模拟，其中球体被 Stiefel 流形代替。我们对 ODL 和 RSR 问题进行的广泛数值实验的结果证明了我们提出的方法的效率和功效。