This article presents the concept of questionnaire experience (QX), intending to add a new element to the psychometric evaluation of questionnaires, which may eventually help increase the validity and reliability of instruments. The application of QX is demonstrated in the development of the Hybrid System Usability Scale (H-SUS), making use of items comprising pictorial and verbal elements to measure perceived usability. The H-SUS was modelled on the verbal version of the System Usability Scale (SUS). Since previous research showed advantages of pictorial scales over verbal scales (e.g., higher respondent motivation) but also disadvantages (e.g., longer completion times), we assumed that hybrid scales would combine the advantages of both scale types. The goal of this study was to compare the two instruments by assessing traditional psychometric criteria (convergent, divergent and criterion-related validity, reliability and sensitivity) and respondent-related aspects of QX (respondent workload, respondent motivation, questionnaire preference, and questionnaire completion time). An online experiment was carried out (N = 152), in which participants interacted with a smartphone prototype and subsequently completed the verbal SUS together with the H-SUS. Results indicate good psychometric properties of the H-SUS. Compared to the SUS, the H-SUS showed similar workload levels for questionnaire completion, higher levels of respondent motivation, but longer questionnaire completion time. Overall, the H-SUS is considered a promising alternative for the evaluation of perceived usability. Finally, QX can be considered a useful concept for identifying potential problems of psychometric instruments in a respondent-centred way, which may help improve the quality of future scales.

本文介绍了问卷体验（QX）的概念，旨在为问卷的心理评估添加一个新元素，这最终可能有助于提高工具的有效性和可靠性。QX的应用在混合系统可用性量表（H-SUS）的开发中得到了证明，该量表使用包含图形和语言元素的项目来衡量感知的可用性。H-SUS以系统可用性量表（SUS）的语言版本为模型。由于先前的研究显示了图形量表优于言语量表的优势（例如，较高的受访者动机），但也有劣势（例如，较长的完成时间），因此我们假设混合量表将结合两种量表的优势。这项研究的目的是通过评估传统的心理测量标准（收敛，发散和与标准相关的有效性，可靠性和敏感性）和与QX的与受访者相关的方面（受访者工作量，受访者动机，问卷偏好和问卷完成）来比较这两种工具时间）。进行了在线实验（N  = 152），其中参与者与智能手机原型进行了交互，随后与H-SUS一起完成了语言SUS。结果表明，H-SUS具有良好的心理测量特性。与SUS相比，H-SUS的问卷完成工作量相似，受访者的动机水平更高，但问卷完成时间更长。总体而言，H-SUS被认为是评估感知可用性的有希望的替代方法。最后，QX被认为是一种以受访者为中心的方式来识别心理测量仪器潜在问题的有用概念，这可能有助于提高未来量表的质量。