In big data scenarios, matrix factorization (MF) is widely used in recommendation systems as it can offer high accuracy and scalability. However, when using MF to process large-scale implicit feedback data, the following two problems arise. One is that it is difficult to effectively obtain negative feedback information, which causes relatively poor recommendation accuracy. The other is that the limited resources of a single machine make the model training inefficient, and in particular, the acquisition of negative feedback information further increases the time complexity of model training. In order to solve the above two problems, we first propose a user-activity and item-popularity weighted matrix factorization (UIWMF) recommendation algorithm, which assigns every missing data different weight based on user activity and item popularity, gets negative feedback information more realistically, and leads to better recommendation accuracy. Meanwhile, in order to reduce the additional computational overhead caused by the weight strategy, we develop a fast optimization strategy to enhance the efficiency. In order to break the resource constraints of a single machine, we propose a distributed UIWMF (DUIWMF) algorithm based on Spark, which adopts an efficient parallel learning algorithm to train the model and utilizes cached in-block and out-block information to effectively reduce the communication overhead in a distributed environment. We conduct experiments on three public datasets, and the experimental results demonstrate that, comparing with the baseline MF methods, DUIWMF model has comparable performance in terms of recommendation accuracy and model training efficiency.

中文翻译：

---

基于快速优化的分布式矩阵分解隐式反馈推荐

在大数据场景中，矩阵分解（MF）在推荐系统中被广泛使用，因为它可以提供高精度和可扩展性。但是，在使用MF处理大规模隐式反馈数据时，会出现以下两个问题。一是难以有效获取负反馈信息，导致推荐准确率相对较差。另一个是单机资源有限，使得模型训练效率低下，尤其是负反馈信息的获取进一步增加了模型训练的时间复杂度。为了解决以上两个问题，我们首先提出了一种用户活跃度和项目流行度加权矩阵分解（UIWMF）推荐算法，该算法根据用户活跃度和项目流行度为每个缺失数据分配不同的权重，更真实地获取负反馈信息，从而获得更好的推荐准确度。同时，为了减少权重策略带来的额外计算开销，我们开发了一种快速优化策略来提高效率。为了打破单机的资源限制，我们提出了一种基于Spark的分布式UIWMF（DUIWMF）算法，该算法采用高效的并行学习算法训练模型，并利用缓存的块内和块外信息有效减少分布式环境中的通信开销。我们在三个公共数据集上进行了实验，实验结果表明，与基线 MF 方法相比，DUIWMF 模型在推荐精度和模型训练效率方面具有可比的性能。