Service-oriented applications commonly involve high-dimensional and sparse (HiDS) interactions among users and service-related entities, e.g., user-item interactions from a personalized recommendation services system. How to perform precise and efficient representation learning on such HiDS interactions data is a hot yet thorny issue. An efficient approach to it is latent factor analysis (LFA), which commonly depends on large-scale non-convex optimization. Hence, it is vital to implement an LFA model able to approximate second-order stationary points efficiently for enhancing its representation learning ability. However, existing second-order LFA models suffer from high computational cost, which significantly reduces its practicability. To address this issue, this paper presents a Momentum-accelerated Hessian-vector algorithm (MH) for precise and efficient LFA on HiDS data. Its main ideas are two-fold: a) adopting the principle of a Hessian-vector-product-based method to utilize the second-order information without manipulating a Hessian matrix directly, and b) incorporating a generalized momentum method into its parameter learning scheme for accelerating its convergence rate to a stationary point. Experimental results on nine industrial datasets demonstrate that compared with state-of-the-art LFA models, an MH-based LFA model achieves gains in both accuracy and convergence rate. These positive outcomes also indicate that a generalized momentum method is compatible with the algorithms, e.g., a second-order algorithm, which implicitly rely on gradients.

中文翻译：

---

基于动量加速 Hessian 向量的潜在因子分析模型


面向服务的应用通常涉及用户和服务相关实体之间的高维稀疏（HiDS）交互，例如来自个性化推荐服务系统的用户-项目交互。如何对此类 HiDS 交互数据进行精确、高效的表示学习是一个热点而又棘手的问题。一种有效的方法是潜在因子分析（LFA），它通常依赖于大规模非凸优化。因此，实现能够有效逼近二阶驻点的 LFA 模型对于增强其表示学习能力至关重要。然而，现有的二阶LFA模型计算成本较高，大大降低了其实用性。为了解决这个问题，本文提出了一种动量加速 Hessian 向量算法 (MH)，可在 HiDS 数据上实现精确高效的 LFA。其主要思想有两个：a）采用基于Hessian向量积的方法的原理来利用二阶信息，而不直接操作Hessian矩阵；b）将广义动量方法纳入其参数学习方案中加速其收敛速度到静止点。在九个工业数据集上的实验结果表明，与最先进的 LFA 模型相比，基于 MH 的 LFA 模型在准确性和收敛速度方面均取得了进步。这些积极的结果还表明，广义动量方法与隐式依赖于梯度的算法（例如二阶算法）兼容。