The evaluation of recommender systems is an area with unsolved questions at several levels. Choosing the appropriate evaluation metric is one of such important issues. Ranking accuracy is generally identified as a prerequisite for recommendation to be useful. Ranking metrics have been adapted for this purpose from the Information Retrieval field into the recommendation task. In this article, we undertake a principled analysis of the robustness and the discriminative power of different ranking metrics for the offline evaluation of recommender systems, drawing from previous studies in the information retrieval field. We measure the robustness to different sources of incompleteness that arise from the sparsity and popularity biases in recommendation. Among other results, we find that precision provides high robustness while normalized discounted cumulative gain offers the best discriminative power. In dealing with cold users, we also find that the geometric mean is more robust than the arithmetic mean as aggregation function over users.

中文翻译：

---

评估前N名推荐中的排名指标

推荐系统的评估是一个在几个层面上尚未解决的问题。选择适当的评估指标是此类重要问题之一。排名准确性通常被认为是推荐有用的前提。为此，已将排名指标从“信息检索”字段修改为推荐任务。在本文中，我们根据信息检索领域的先前研究，对推荐系统的脱机评估的不同排名指标的鲁棒性和判别力进行了原则性分析。我们评估了由于推荐中的稀疏性和受欢迎度偏差而导致的对不完整来源的鲁棒性。在其他结果中，我们发现，精度提供了很高的鲁棒性，而归一化折现后的累积增益则提供了最佳的判别能力。在处理冷用户时，我们还发现，几何平均值比算术平均值更健壮，因为它是针对用户的聚合函数。