Recommender systems are frequently evaluated using performance indexes based on variants and extensions of precision-like measures. As these measures are biased toward popular items, a list of recommendations simply must include a few popular items to perform well. To address the popularity bias challenge, new approaches for novelty and diversity evaluation have been proposed. On the one hand, novelty-based approaches model the quality of being new as apposed to that which is already known. Novelty approaches are commonly based on item views or user rates. On the other hand, diversity approaches model the quality of an item that is composed of different content elements. Diversity measures are commonly rooted in content-based features that characterize the diversity of the content of an item in terms of the presence/absence of a number of predefined nuggets of information. As item contents are also biased to popular contents (e.g., drama in movies or pop in music), diversity-based measures are also popularity biased. To alleviate the effect of popularity bias on diversity measures, we used an evaluation approach based on the degree of novelty of the elements that make up each item. We named this approach content novelty, as it mixes content and diversity approaches in a single and coherent evaluation framework. Experimental results show that our proposal is feasible and useful. Our findings demonstrate that the proposed measures yield consistent and interpretable results, producing insights that reduce the impact of popularity bias in the evaluation of recommender systems.

中文翻译：

---

评估推荐系统中的内容新颖性

推荐系统经常使用基于类似精确度量的变体和扩展的性能指标进行评估。由于这些措施偏向于热门项目，因此推荐列表必须包括一些热门项目才能表现良好。为了解决流行偏见的挑战，已经提出了新颖性和多样性评估的新方法。一方面，基于新颖性的方法将新的质量建模为与已知质量相对应的质量。新颖性方法通常基于项目浏览量或用户率。另一方面，多样性方法对由不同内容元素组成的项目的质量进行建模。多样性度量通常植根于基于内容的特征，这些特征根据多个预定义信息块的存在/不存在来表征项目内容的多样性。由于项目内容也偏向于流行内容（例如，电影中的戏剧或音乐中的流行），基于多样性的措施也偏向于流行性。为了减轻流行度偏差对多样性度量的影响，我们使用了一种基于构成每个项目的元素的新颖程度的评估方法。我们将这种方法命名为内容新颖性，因为它在一个单一且连贯的评估框架中混合了内容和多样性方法。实验结果表明，我们的建议是可行和有用的。我们的研究结果表明，提议的措施产生了一致且可解释的结果，