A large portion of the cost of any software lies in the time spent by developers in understanding a program’s source code before any changes can be undertaken. Measuring program comprehension is not a trivial task. In fact, different studies use self-reported and various psycho-physiological measures as proxies. In this research, we propose a methodology using functional Near Infrared Spectroscopy (fNIRS) and eye tracking devices as an objective measure of program comprehension that allows researchers to conduct studies in environments close to real world settings, at identifier level of granularity. We validate our methodology and apply it to study the impact of lexical, structural, and readability issues on developers’ cognitive load during bug localization tasks. Our study involves 25 undergraduate and graduate students and 21 metrics. Results show that the existence of lexical inconsistencies in the source code significantly increases the cognitive load experienced by participants not only on identifiers involved in the inconsistencies but also throughout the entire code snippet. We did not find statistical evidence that structural inconsistencies increase the average cognitive load that participants experience, however, both types of inconsistencies result in lower performance in terms of time and success rate. Finally, we observe that self-reported task difficulty, cognitive load, and fixation duration do not correlate and appear to be measuring different aspects of task difficulty.

中文翻译：

---

在错误定位期间测量词汇和结构不一致对开发人员认知负荷的影响

任何软件的大部分成本都在于开发人员在进行任何更改之前理解程序源代码所花费的时间。衡量程序理解能力并不是一项微不足道的任务。事实上，不同的研究使用自我报告和各种心理生理措施作为代理。在这项研究中，我们提出了一种方法，使用功能性近红外光谱 (fNIRS) 和眼动追踪设备作为程序理解的客观衡量标准，使研究人员能够在接近现实世界设置的环境中，在标识符级别进行研究。我们验证了我们的方法并将其应用于研究词汇、结构和可读性问题在错误定位任务期间对开发人员认知负荷的影响。我们的研究涉及 25 名本科生和研究生以及 21 个指标。结果表明，源代码中词汇不一致的存在显着增加了参与者的认知负荷，不仅涉及不一致的标识符，而且贯穿整个代码片段。我们没有发现统计证据表明结构不一致会增加参与者经历的平均认知负荷，但是，两种类型的不一致都会导致在时间和成功率方面的表现较低。最后，我们观察到自我报告的任务难度、认知负荷和注视持续时间不相关，似乎在衡量任务难度的不同方面。