The Ruff Figural Fluency Test (RFFT) is a valid but time-consuming and labour-intensive cognitive paper-and-pencil test. A digital RFFT was developed that can be conducted independently using an iPad and Apple Pencil and RFFT scores are computed automatically. We investigated the validity and reliability of this digital RFFT. We randomly allocated participants to the digital or paper-and-pencil RFFT. After the first test, the other test was performed immediately (cross-over). Participants were invited for a second digital RFFT 1 week later. For the digital RFFT, an (automatic) algorithm and two independent raters (criterion standard) assessed the number of unique designs (UD) and perseverative errors (PE). These raters also assessed the paper-and-pencil RFFT. We used Intraclass correlation coefficients (ICC), sensitivity, specificity, %-agreement, Kappa, and Bland–Altman plots. We included 94 participants (mean (SD) age 39.9 (14.8), 73.4% follow-up). Mean (SD) UD and median (IQR) PE of the digital RFFT were 84.2 (26.0) and 4 (2–7.3), respectively. Agreement between manual and automatic scoring of the digital RFFT was high for UD (ICC = 0.99, 95% CI 0.98, 0.99, sensitivity = 0.98; specificity = 0.96) and PE (ICC = 0.99, 95% CI 0.98, 0.99; sensitivity = 0.90, specificity = 1.00), indicating excellent criterion validity. Small but significant differences in UD were found between the automatic and manual scoring (mean difference: − 1.12, 95% CI − 1.92, − 0.33). Digital and paper-and-pencil RFFT had moderate agreement for UD (ICC = 0.73, 95% CI 0.34, 0.87) and poor agreement for PE (ICC = 0.47, 95% CI 0.30, 0.62). Participants had fewer UD on the digital than paper-and-pencil RFFT (mean difference: − 7.09, 95% CI − 11.80, − 2.38). The number of UD on the digital RFFT was associated with higher education (Spearman’s r = 0.43, p < 0.001), and younger age (Pearson’s r = − 0.36, p < 0.001), showing its ability to discriminate between different age categories and levels of education. Test–retest reliability was moderate (ICC = 0.74, 95% CI 0.61, 0.83). The automatic scoring of the digital RFFT has good criterion and convergent validity. There was low agreement between the digital RFFT and paper-and-pencil RFFT and moderate test–retest reliability, which can be explained by learning effects. The digital RFFT is a valid and reliable instrument to measure executive cognitive function among the general population and is a feasible alternative to the paper-and-pencil RFFT in large-scale studies. However, its scores cannot be used interchangeably with the paper-and-pencil RFFT scores.

中文翻译：

---

数字Ruff图形流利性测试（RFFT）的有效性和可靠性

Ruff图形流利度测试（RFFT）是有效但费时且费力的认知纸笔测试。开发了一种数字RFFT，可以使用iPad和Apple Pencil进行独立操作，并自动计算RFFT分数。我们研究了这种数字RFFT的有效性和可靠性。我们随机分配参与者参加数字或纸笔RFFT。第一次测试后，立即进行另一个测试（交叉测试）。1周后邀请参与者进行第二次数字RFFT。对于数字RFFT，一种（自动）算法和两个独立的评分器（标准标准）评估了独特设计（UD）和持久误差（PE）的数量。这些评估者还评估了纸笔RFFT。我们使用了类内相关系数（ICC），敏感性，特异性，百分比一致性，Kappa和Bland–Altman图。我们纳入了94名参与者（平均（SD）年龄39.9（14.8），随访73.4％）。数字RFFT的平均值（SD）UD和中位数（IQR）PE分别为84.2（26.0）和4（2-7.3）。对于UD（ICC = 0.99，95％CI 0.98，0.99，灵敏度= 0.98;特异性= 0.96）和PE（ICC = 0.99，95％CI 0.98，0.99;灵敏度= 0.90，特异性= 1.00），表明标准的有效性极好。自动评分和手动评分之间的UD差异很小但很明显（均值差异：-1.12，95％CI-1.92，-0.33）。数字和纸笔RFFT对UD的一致性中等（ICC = 0.73，95％CI 0.34，0.87），而对PE的一致性较差（ICC = 0.47，95％CI 0.30，0.62）。与数字铅笔RFFT相比，参与者在数字上的UD更少（平均差异：− 7.09，95％CI − 11.80，− 2.38）。数字RFFT上的UD数量与高等教育（Spearman的r = 0.43，p <0.001）和较年轻的年龄（Pearson的r = − 0.36，p <0.001）相关，表明了其区分不同年龄类别和水平的能力教育之中。重测信度是中等的（ICC = 0.74，95％CI 0.61，0.83）。数字RFFT的自动评分具有良好的判据和收敛的有效性。数字RFFT和纸笔RFFT与适度的重测信度之间的一致性较低，这可以通过学习效果来解释。数字RFFT是一种有效且可靠的工具，可以测量普通人群中的执行认知功能，并且是大规模研究中纸笔RFFT的可行替代方案。但是，其分数不能与纸笔RFFT分数互换使用。