Many information needs cannot be resolved with a single query, and instead lead naturally to a sequence of queries, issued as a search session. In a session test collection, each topic has an associated query sequence, with users assumed to follow that sequence when reformulating their queries. Here we propose a session-based offline evaluation framework as an extension to the existing query-based C/W/L framework, and use that framework to devise an adaptive session-based effectiveness metric, as a way of measuring the overall usefulness of a search session. To realize that goal, data from two commercial search engines is employed to model the two required behaviors: the user conditional continuation probability, and the user conditional reformulation probability. We show that the session-extended C/W/L framework allows the development of new metrics with associated user models that give rise to greater correlation with observed user behavior during search sessions than do previous session metrics, and hence provide a richer context in which to compare retrieval systems at a session level.

单个查询无法解决许多信息需求，而是自然导致了一系列查询，这些查询是作为搜索会话发出的。在会话测试集合中，每个主题都有一个关联的查询序列，假设用户在重新制定其查询时遵循该序列。在这里，我们提出了一个基于会话的离线评估框架，作为对现有基于查询的C / W / L框架的扩展，并使用该框架设计出一种自适应的基于会话的有效性指标，作为衡量某项服务总体有效性的一种方式搜索会话。为了实现该目标，采用了来自两个商业搜索引擎的数据来对两个必需的行为进行建模：用户条件延续概率和用户条件重构概率。我们显示，会话扩展的C / W / L框架允许开发具有关联用户模型的新指标，从而比以前的会话指标与搜索会话期间观察到的用户行为具有更大的相关性，从而提供了更丰富的上下文在会话级别比较检索系统。