Recommender systems’ evaluation is usually based on predictive accuracy and information retrieval metrics, with better scores meaning recommendations are of higher quality. However, new algorithms are constantly developed and the comparison of results of algorithms within an evaluation framework is difficult since different settings are used in the design and implementation of experiments. In this paper, we propose a guidelines-based approach that can be followed to reproduce experiments and results within an evaluation framework. We have evaluated our approach using a real dataset, and well-known recommendation algorithms and metrics; to show that it can be difficult to reproduce results if certain settings are missing, thus resulting in more evaluation cycles required to identify the optimal settings.

中文翻译：

---

一种基于指南的方法，用于协助推荐系统评估中实验的可重复性

推荐系统的评估通常基于预测准确性和信息检索指标，分数越高意味着推荐的质量越高。然而，新算法不断开发，并且在评估框架内比较算法结果是困难的，因为在实验的设计和实施中使用了不同的设置。在本文中，我们提出了一种基于指南的方法，可以遵循该方法在评估框架内重现实验和结果。我们使用真实的数据集以及著名的推荐算法和指标评估了我们的方法；表明如果缺少某些设置，可能难以重现结果，从而导致需要更多的评估周期来确定最佳设置。