Sketching is a dimensionality reduction technique where one compresses a matrix by linear combinations that are typically chosen at random. A line of work has shown how to sketch the Hessian to speed up each iteration in a second order method, but such sketches usually depend only on the matrix at hand, and in a number of cases are even oblivious to the input matrix. One could instead hope to learn a distribution on sketching matrices that is optimized for the specific distribution of input matrices. We show how to design learned sketches for the Hessian in the context of second order methods, where we learn potentially different sketches for the different iterations of an optimization procedure. We show empirically that learned sketches, compared with their "non-learned" counterparts, improve the approximation accuracy for important problems, including LASSO, SVM, and matrix estimation with nuclear norm constraints. Several of our schemes can be proven to perform no worse than their unlearned counterparts. Additionally, we show that a smaller sketching dimension of the column space of a tall matrix is possible, assuming an oracle for predicting rows which have a large leverage score.

中文翻译：

---

粗麻布学习增强素描

草图绘制是一种降维技术，其中，人们通常通过随机选择的线性组合来压缩矩阵。一条工作线显示了如何绘制Hessian草图以加快二阶方法的每次迭代速度，但是这种草图通常仅取决于手头的矩阵，并且在许多情况下甚至忽略了输入矩阵。取而代之的是，人们希望学习素描矩阵的分布，该分布针对输入矩阵的特定分布进行了优化。我们展示了如何在二阶方法的背景下为Hessian设计学到的草图，在其中我们可以为优化过程的不同迭代学习潜在的不同草图。我们通过经验证明，与“非学习”的草图相比，学过的草图更容易理解，提高重要问题的近似精度，包括LASSO，SVM和具有核规范约束的矩阵估计。可以证明我们的几种方案的性能不比未经学习的方案好。另外，我们表明，假设使用预言机来预测具有较大杠杆得分的行，则可以用较小的草图绘制高个矩阵的列空间。