QueryCrumbs search query history visualization – Usability, transparency and long-term usage

Abstract

Models of human information seeking reveal that search, in particular ad-hoc retrieval, is non-linear and iterative. Despite these findings, today’s search user interfaces do not support non-linear navigation, like for example backtracking in time. We propose QueryCrumbs, a compact and easy-to-understand visualization for navigating the search query history supporting iterative query refinement. We apply a multi-layered interface design to support novices and first-time users as well as intermediate and expert users. The visualization is evaluated with novice users in a formative user study, with experts in a think aloud test and its usage in a long-term study with software logging. The formative evaluation showed that the interactions can be easily performed, and the visual encodings were well understood without instructions. Results indicate that QueryCrumbs can support users when searching for information in an iterative manner. The evaluation with experts showed that expert users can gain valuable insights into the back-end search engine by identifying specific patterns in the visualization. In a long-term usage study, we observed an uptake of the visualization, indicating that users deem QueryCrumbs beneficial for their search interactions.

Introduction

A common phenomenon in Web search is that users re-access Web resources that have been found in the past. Accessing previously found information is different from information seeking, e.g., by being more targeted and more directed involving recognition and recall activities [1]. While active strategies (i.e., explicit storage of the information) would support information refinding, passive strategies with no explicit storage are much more common, especially when search tasks are interrupted [2]. Such passive strategies require to recall how or where the information was found previously. The difficulty of recalling where and how information on the Web was accessed is known as the “Lost in Hyperspace syndrome” [3].

While strictly speaking the “Lost in Hyperspace syndrome” refers to the navigation of hypermedia only, an analysis of human information seeking models shows a similar behavior in the context of Web search. Models of human information seeking describe and structure the way humans search for information in an information source (for an overview see [4]). These models define human information seeking as an iterative process in which query reformulation is a common step (e.g., [5], [6]). Usually, multiple steps have to be taken and multiple query reformulations are necessary before the information need is fully satisfied. The demand to include a search history for supporting the query reformulation stage has been explicitly stated (e.g.,[4], [7]). A search history also supports information re-finding for interrupted search sessions, which have been found to occur in 40% of all information seeking tasks in the study of Sellen et al. [8]. Resuming a search from a previous query relying on human memory has been shown to be only accurate in 72% of the time [9]. In the information retrieval community, automatic query refinement is an active research topic (e.g. [10]). This indicates that, in general, queries posed by users are underspecified and need to be refined iteratively.

We propose QueryCrumbs, a simple-to-understand, compact visualization for accessing, altering, and resubmitting previously issued queries. The concept is similar to bread crumbing interfaces as navigational aid for web sites [11]. Fig. 1 shows the conceptual idea of the QueryCrumbs visualization. Each query is represented by a mark, the position of the mark indicates the position of the query in the sequence of queries and different notions of query similarity are encoded in the mark’s visual attributes. We introduce three different measures for query similarity to capture the general relationship between queries and corresponding mappings to the marks’ visual attributes. The similarity is measured on different levels of detail, suitable for different user groups and tasks. In order to evaluate the usefulness of this visual representation, we pursue a layered interface design approach [12] introducing different notions of similarity in each layer. We evaluate the visualization and interaction design in a formative user study with novices. Additionally, we performed a think aloud test with experts in information retrieval to investigate which conclusion can be drawn about the search engine when using the visualization with the advanced similarity encoding. Finally, we evaluate the actual usage of QueryCrumbs outside the lab environment with software logging in a long-term study. Concretely, the contribution of this paper is as follows:

• We introduce a human querying model as the conceptual basis for search history visualizations.

• We propose QueryCrumbs, a search engine agnostic, compact and interactive visualization supporting overview and navigation of the query history

• We account for universal usability by applying the multi-layered user interface design method to the design of the visualization.

• We show, that QueryCrumbs is usable by search lay persons without instructions, search experts can gain valuable insights into search engine internals (transparency) and demonstrate an uptake outside the lab environment.

This paper extends the study in [13]¹ and builds upon the underlying human querying model and basic visualization concepts in previous work [14]². We unify those two papers into a joint representation for self-containedness, extend the discussion of the multi-layered approach and present an evaluation ”in the wild“. As other tools, search history visualizations need to be both, useful and usable. In own previous work, a formative user study showed that QueryCrumbs was deemed usable and useful for search lay persons [14]. Further own previous work showed – again in a temporally constrained lab study – that encoding of additional information in QueryCrumbs enabled search experts to gain understanding of the search backend internals [13]. These results encouraged an out-of-lab implementation of QueryCrumbs: Proving usability and usefulness in a lab environment does not tell, whether people are actually using the visualization outside the lab and if so, in which way. To close this gap, we present an out-of-lab long-term study with a widely used search backend (Google Scholar) in this paper. This crucial extension complements our previous work (proving usability and usefulness) with insights to the actual usage of QueryCrumbs.

The remainder of the paper is organized as follows: After discussing related work, we describe the human querying model in Section 3 and from that derive the conceptual idea for the visualization in Section 4. Then, the multi-layered approach to visualization and interaction design is explained in detail in Section 5. The formative user evaluation described in Section 6 assesses the usability for the two layers designed for first-time and intermediate users. In a study with experts we then assess which conclusions can be drawn about the underlying search engine with the help of the visualization (cf. Section 7). We evaluate the actual usage of the visualization in Section 8 and conclude the paper with a discussion of design decisions and their consequences, limitations and an outlook on future work.