Simulating online recommender system performance is notoriously difficult and the discrepancy between the online and offline behaviors is typically not accounted for in offline evaluations. Recommender systems research tends to evaluate model performance on randomly sampled targets, yet the same systems are later used to predict user behavior sequentially from a fixed point in time. This disparity permits weaknesses to go unnoticed until the model is deployed in a production setting. We first demonstrate how omitting temporal context when evaluating recommender system performance leads to false confidence. To overcome this, we propose an offline evaluation protocol modeling the real-life use-case that simultaneously accounts for temporal context. Next, we propose a training procedure to further embed the temporal context in existing models: we introduce it in a multi-objective approach to traditionally time-unaware recommender systems. We confirm the advantage of adding a temporal objective via the proposed evaluation protocol. Finally, we validate that the Pareto Fronts obtained with the added objective dominate those produced by state-of-the-art models that are only optimized for accuracy on three real-world publicly available datasets. The results show that including our temporal objective can improve recall@20 by up to 20%.

中文翻译：

---

推荐系统中的在线行为建模：时间上下文的重要性

模拟在线推荐系统的性能是出了名的困难，在线和离线行为之间的差异通常不会在离线评估中考虑在内。推荐系统研究倾向于在随机采样的目标上评估模型性能，但稍后使用相同的系统从固定时间点按顺序预测用户行为。这种差异使得弱点在生产环境中部署模型之前不会被注意到。我们首先演示了在评估推荐系统性能时省略时间上下文如何导致错误的置信度。为了克服这个问题，我们提出了一种离线评估协议，该协议对现实生活中的用例进行建模，同时考虑了时间上下文。接下来，我们提出了一个训练程序，以进一步将时间上下文嵌入现有模型中：我们以多目标方法将其引入到传统的时间不感知推荐系统中。我们确认通过提议的评估协议添加时间目标的优势。最后，我们验证了通过添加目标获得的帕累托前沿主导了由仅针对三个真实世界公开可用数据集的准确性优化的最先进模型产生的帕累托前沿。结果表明，包括我们的时间目标可以将召回@20 提高多达 20%。我们验证了通过附加目标获得的帕累托前沿主导了由仅针对三个真实世界公开可用数据集的准确性优化的最先进模型产生的帕累托前沿。结果表明，包括我们的时间目标可以将召回@20 提高多达 20%。我们验证了通过附加目标获得的帕累托前沿主导了由仅针对三个真实世界公开可用数据集的准确性优化的最先进模型产生的帕累托前沿。结果表明，包括我们的时间目标可以将召回@20 提高多达 20%。