This paper provides overview and instruction regarding the evaluation of interactive information retrieval systems with users. The primary goal of this article is to catalog and compile material related to this topic into a single source. This article (1) provides historical background on the development of user-centered approaches to the evaluation of interactive information retrieval systems; (2) describes the major components of interactive information retrieval system evaluation; (3) describes different experimental designs and sampling strategies; (4) presents core instruments and data collection techniques and measures; (5) explains basic data analysis techniques; and (4) reviews and discusses previous studies. This article also discusses validity and reliability issues with respect to both measures and methods, presents background information on research ethics and discusses some ethical issues which are specific to studies of interactive information retrieval (IIR). Finally, this article concludes with a discussion of outstanding challenges and future research directions.

本文提供了有关与用户进行交互式信息检索系统评估的概述和说明。本文的主要目标是将与该主题相关的材料编目并汇编到一个单一的来源中。本文（1）提供了以用户为中心的交互式信息检索系统评估方法发展的历史背景；（2）描述了交互式信息检索系统评估的主要组成部分；（3）描述了不同的实验设计和采样策略；（4）介绍了核心手段和数据收集技术与措施；（5）说明基本数据分析技术；（4）回顾和讨论以前的研究。本文还讨论了有关度量和方法的有效性和可靠性问题，提供有关研究伦理的背景信息，并讨论一些特定于交互式信息检索（IIR）研究的伦理问题。最后，本文最后讨论了突出的挑战和未来的研究方向。