How can we compute the pseudoinverse of a sparse feature matrix efficiently and accurately for solving optimization problems? A pseudoinverse is a generalization of a matrix inverse, which has been extensively utilized as a fundamental building block for solving linear systems in machine learning. However, an approximate computation, let alone an exact computation, of pseudoinverse is very time-consuming due to its demanding time complexity, which limits it from being applied to large data. In this paper, we propose FastPI (Fast PseudoInverse), a novel incremental singular value decomposition (SVD) based pseudoinverse method for sparse matrices. Based on the observation that many real-world feature matrices are sparse and highly skewed, FastPI reorders and divides the feature matrix and incrementally computes low-rank SVD from the divided components. To show the efficacy of proposed FastPI, we apply them in real-world multi-label linear regression problems. Through extensive experiments, we demonstrate that FastPI computes the pseudoinverse faster than other approximate methods without loss of accuracy. %and uses much less memory compared to full-rank SVD based approach. Results imply that our method efficiently computes the low-rank pseudoinverse of a large and sparse matrix that other existing methods cannot handle with limited time and space.

中文翻译：

---

使用稀疏矩阵重新排序和增量方法快速准确的伪逆

我们如何有效且准确地计算稀疏特征矩阵的伪逆以解决优化问题？伪逆是矩阵逆的推广，它已被广泛用作解决机器学习中线性系统的基本构建块。然而，伪逆的近似计算，更不用说精确计算了，由于其要求的时间复杂度，这限制了其应用于大数据，因此非常耗时。在本文中，我们提出了 FastPI (Fast PseudoInverse)，这是一种新的基于增量奇异值分解 (SVD) 的稀疏矩阵伪逆方法。基于对现实世界中许多特征矩阵稀疏且高度偏斜的观察，FastPI 对特征矩阵进行重新排序和划分，并从划分的组件中增量计算低秩 SVD。为了展示提出的 FastPI 的功效，我们将它们应用于现实世界的多标签线性回归问题。通过大量实验，我们证明 FastPI 比其他近似方法更快地计算伪逆而不会损失准确性。% 并且与基于全秩 SVD 的方法相比使用更少的内存。结果表明，我们的方法有效地计算了其他现有方法无法在有限的时间和空间内处理的大型稀疏矩阵的低秩伪逆。我们证明 FastPI 比其他近似方法更快地计算伪逆而不会损失准确性。% 并且与基于全秩 SVD 的方法相比使用更少的内存。结果表明，我们的方法有效地计算了其他现有方法无法在有限的时间和空间内处理的大型稀疏矩阵的低秩伪逆。我们证明 FastPI 比其他近似方法更快地计算伪逆而不会损失准确性。% 并且与基于全秩 SVD 的方法相比使用更少的内存。结果表明，我们的方法有效地计算了其他现有方法无法在有限的时间和空间内处理的大型稀疏矩阵的低秩伪逆。