Abstract In search of the silver bullet to solve the password problem, the field of knowledge-based authentication has become bloated with novel proposals aiming to replace textual passwords. The emphasis on the quantity of studies as opposed to the quality of evaluation has made it difficult to compare the methods, as well as to validate and generalize the results. To improve the quality of security and usability evaluations, experimental design decisions should be reviewed and standardized. In this systematic review, we focus on the evaluation of the shoulder surfing attack (SSA) vulnerability. We formulate two research questions to help us determine how the design of the method should affect the SSA experimental design process, and how different design decisions affect the validity and interpretability of the results under various assumptions and threat models. To provide the researchers with comprehensive literature on SSA evaluation, we identify empirical shoulder surfing studies conforming to a predefined set of quality criteria. Based on the design features extracted from the experiments, we develop an evaluation framework for the assessment of the shoulder surfing experimental setup. In the follow-up analysis, we assess the proposed methods’ design features, and the quality of their SSA experiments, using Schaub et al.’s design aspect and our SSA evaluation frameworks, respectively. Through exhaustive analysis, we strive to streamline and standardize experimental decisions by showcasing their impact on the outcome of the study, and generate guidelines for a more objective design of shoulder surfing experiments.

中文翻译：

---

肩部冲浪实验：系统文献综述

摘要 为了寻找解决密码问题的灵丹妙药，基于知识的身份验证领域已经变得臃肿，新的提议旨在取代文本密码。强调研究的数量而不是评估的质量，这使得比较这些方法以及验证和概括结果变得困难。为了提高安全性和可用性评估的质量，应审查和标准化实验设计决策。在本系统综述中，我们重点评估肩部冲浪攻击 (SSA) 漏洞。我们制定了两个研究问题来帮助我们确定方法的设计应该如何影响 SSA 实验设计过程，以及在各种假设和威胁模型下，不同的设计决策如何影响结果的有效性和可解释性。为了向研究人员提供关于 SSA 评估的全面文献，我们确定了符合预定义质量标准的经验性肩部冲浪研究。基于从实验中提取的设计特征，我们开发了一个评估框架，用于评估肩部冲浪实验装置。在后续分析中，我们分别使用 Schaub 等人的设计方面和我们的 SSA 评估框架来评估所提出方法的设计特征和 SSA 实验的质量。通过详尽的分析，我们努力通过展示实验决策对研究结果的影响来简化和标准化实验决策，