ABSTRACT

Objective: This experimental study assessed the effects of an Apple iPad (Apple and iPad are trademarks of Apple Inc., registered in the U.S. and other countries) grading instrument on airline evaluator performance.

Background: Extant research exploring the effects of grading formats on evaluator performance focused on pen and paper modalities. This research investigated how a contemporary iPad-based grading format affected airline evaluator performance.

Method: Forty-five evaluators from a major U.S. airline graded a Boeing 767 (Boeing is a wordmark of The Boeing Company, registered in the U.S. and other countries) flight crew experiencing an engine failure during takeoff. Three grading formats were used: pen and paper, a paper-based equivalent of the iPad grading format designed to replicate the function of the iPad, and the iPad. The effects of the grading formats were measured using the following dependent variables: recorded technical criteria, recorded non-technical criteria, and correlation and inter-rater agreement with a referent score.

Results: Members of the iPad paper-based equivalent and iPad groups recorded significantly more technical and non-technical criteria compared to participants using pen and paper (p < .001). Members of the iPad paper-based equivalent and iPad groups had significantly higher correlation (p = .01) and inter-rater agreement levels (p < .01) compared to participants using pen and paper.

Conclusion: The results of the study failed to support the iPad as a superior grading format. Research outcomes suggested structured grading may be a more important factor in predicting airline evaluator performance than the type of grading format used.

摘要

目的：本实验研究评估了Apple iPad（Apple和iPad是Apple Inc.在美国和其他国家/地区注册的商标）分级工具对航空公司评估人员的影响。

背景：现有研究探讨了评分格式对评估者绩效的影响，重点是笔和纸的形式。这项研究调查了当代基于iPad的评分格式如何影响航空公司评估员的表现。

方法：来自美国一家主要航空公司的四十五名评估员将波音767（波音是波音公司的商标，在美国和其他国家注册）的等级评定为机组人员在起飞期间出现发动机故障。使用了三种分级格式：笔和纸，一种基于纸质的iPad分级格式，旨在复制iPad和iPad的功能。评分格式的影响是使用以下因变量来衡量的：记录的技术标准，记录的非技术标准以及相关性和评分者之间的一致以及参考分数。

结果：与使用笔和纸的参与者相比，与纸本相当的iPad和iPad组的成员记录了更多的技术和非技术标准（p <.001）。与使用笔和纸的参与者相比，与纸本相当的iPad组和iPad组的成员具有更高的相关性（p = .01）和评估者之间的协议水平（p <.01）。

结论：研究结果未能支持iPad作为高级评分格式。研究结果表明，与使用的评分格式类型相比，结构化评分可能是预测航空公司评估师绩效的更重要因素。