We study the practical consequences of dataset sampling strategies on the performance of recommendation algorithms. Recommender systems are generally trained and evaluated on samples of larger datasets. Samples are often taken in a naive or ad-hoc fashion: e.g. by sampling a dataset randomly or by selecting users or items with many interactions. As we demonstrate, commonly-used data sampling schemes can have significant consequences on algorithm performance -- masking performance deficiencies in algorithms or altering the relative performance of algorithms, as compared to models trained on the complete dataset. Following this observation, this paper makes the following main contributions: (1) characterizing the effect of sampling on algorithm performance, in terms of algorithm and dataset characteristics (e.g. sparsity characteristics, sequential dynamics, etc.); and (2) designing SVP-CF, which is a data-specific sampling strategy, that aims to preserve the relative performance of models after sampling, and is especially suited to long-tail interaction data. Detailed experiments show that SVP-CF is more accurate than commonly used sampling schemes in retaining the relative ranking of different recommendation algorithms.

中文翻译：

---

SVP-CF：通过代理选择协同过滤数据

我们研究数据集采样策略对推荐算法性能的实际影响。推荐系统通常在较大数据集的样本上进行训练和评估。样本通常以天真或特别的方式获取：例如通过随机采样数据集或通过选择具有许多交互的用户或项目。正如我们所展示的，与在完整数据集上训练的模型相比，常用的数据采样方案会对算法性能产生重大影响——掩盖算法中的性能缺陷或改变算法的相对性能。遵循这一观察，本文做出以下主要贡献：（1）根据算法和数据集特征（例如稀疏特征，顺序动力学等）；(2) 设计 SVP-CF，这是一种数据特定的采样策略，旨在保持采样后模型的相对性能，特别适用于长尾交互数据。详细的实验表明，SVP-CF 在保留不同推荐算法的相对排名方面比常用的采样方案更准确。