Many problems in data science can be treated as estimating a low-rank matrix from highly incomplete, sometimes even corrupted, observations. One popular approach is to resort to matrix factorization, where the low-rank matrix factors are optimized via first-order methods over a smooth loss function, such as the residual sum of squares. While tremendous progress has been made in recent years, the natural smooth formulation suffers from two sources of ill-conditioning, where the iteration complexity of gradient descent scales poorly both with the dimension as well as the condition number of the low-rank matrix. Moreover, the smooth formulation is not robust to corruptions. In this paper, we propose scaled subgradient methods to minimize a family of nonsmooth and nonconvex formulations-in particular, the residual sum of absolute errors-which is guaranteed to converge at a fast rate that is almost dimension-free and independent of the condition number, even in the presence of corruptions. We illustrate the effectiveness of our approach when the observation operator satisfies certain mixed-norm restricted isometry properties, and derive state-of-the-art performance guarantees for a variety of problems such as robust low-rank matrix sensing and quadratic sampling.

中文翻译：

---

使用缩放次梯度方法的低秩矩阵恢复：无需条件数的快速鲁棒收敛


数据科学中的许多问题可以被视为根据高度不完整、有时甚至损坏的观察来估计低秩矩阵。一种流行的方法是采用矩阵分解，其中通过平滑损失函数（例如残差平方和）上的一阶方法来优化低秩矩阵因子。尽管近年来取得了巨大进步，但自然平滑公式受到两个病态来源的影响，其中梯度下降的迭代复杂度与低秩矩阵的维数和条件数的缩放比例都很差。此外，光滑的公式对于腐败并不具有鲁棒性。在本文中，我们提出了缩放次梯度方法来最小化一系列非光滑和非凸公式 - 特别是绝对误差的残差和 - 保证以几乎无量纲且独立于条件数的快速速率收敛，即使存在腐败。我们说明了当观察算子满足某些混合范数限制等距属性时我们的方法的有效性，并为各种问题（例如鲁棒的低秩矩阵传感和二次采样）得出最先进的性能保证。