A statistical procedure is assumed to produce comparable results across programs. Using the case of an exploratory factor analysis procedure—principal axis factoring (PAF) and promax rotation—we show that this assumption is not always justified. Procedures with equal names are sometimes implemented differently across programs: a jingle fallacy. Focusing on two popular statistical analysis programs, we indeed discovered a jingle jungle for the above procedure: Both PAF and promax rotation are implemented differently in the psych R package and in SPSS. Based on analyses with 247 real and 216,000 simulated data sets implementing 108 different data structures, we show that these differences in implementations can result in fairly different factor solutions for a variety of different data structures. Differences in the solutions for real data sets ranged from negligible to very large, with 42% displaying at least one different indicator-to-factor correspondence. A simulation study revealed systematic differences in accuracies between different implementations, and large variation between data structures, with small numbers of indicators per factor, high factor intercorrelations, and weak factors resulting in the lowest accuracies. Moreover, although there was no single combination of settings that was superior for all data structures, we identified implementations of PAF and promax that maximize performance on average. We recommend researchers to use these implementations as best way through the jungle, discuss model averaging as a potential alternative, and highlight the importance of adhering to best practices of scale construction.

假设统计程序可以在不同程序中产生可比较的结果。使用探索性因子分析程序的案例——主轴因子分解 (PAF) 和 promax 旋转——我们表明这种假设并不总是合理的。具有相同名称的程序有时在程序中的实现方式不同：叮当谬误。着眼于两个流行的统计分析程序，我们确实发现了上述过程的叮当丛林：PAF 和 promax 旋转在心理中的实现方式不同R 包和 SPSS。基于对实现 108 种不同数据结构的 247 个真实数据集和 216,000 个模拟数据集的分析，我们表明，这些实现上的差异可能导致针对各种不同数据结构的完全不同的因子解决方案。真实数据集的解决方案差异从可以忽略不计到非常大，42% 的人至少显示出一种不同的指标与因素的对应关系。一项模拟研究揭示了不同实现之间的准确性存在系统性差异，数据结构之间存在较大差异，每个因素的指标数量少，因素相关性高，因素较弱，导致准确性最低。此外，尽管没有一种单一的设置组合对所有数据结构都具有优越性，我们确定了平均最大化性能的 PAF 和 promax 的实现。我们建议研究人员使用这些实现作为穿越丛林的最佳方式，讨论模型平均作为一种潜在的替代方案，并强调坚持规模构建最佳实践的重要性。