Massive Open Online Courses (MOOCs) have gained traction as resources for professional development. This article presents the method that we used to evaluate a professional development MOOC for postdoctoral trainees that was created by a university consortium in the US. Most approaches to evaluating MOOCs focus on analysis of participation, outcomes from course assignments, self-reported learning outcomes, course completion and user pathways through the online content or clickstream data. Few published evaluations describe in detail how learning happens within online courses and the anticipated medium and longer term cognitive and behavioral outcomes on participants. This work aims to guide those who are designing, implementing and evaluating MOOCs through applying theories of change to focus evaluation on the process of learning. This approach can be used as a complement to traditional approaches for evaluating MOOCs. We described how we worked with the MOOC team building the content to develop a theory of change for each module (or lesson) within the MOOC and used the theory of change to guide evaluation of short and medium term participant cognitive, affective and attitudinal, and behavioral outcomes. Finally, we share lessons learned and suggestions for implementing theories of change in both the design and evaluation phases of MOOC development.

大规模开放在线课程 (MOOC) 作为专业发展的资源而备受关注。本文介绍了我们用于评估由美国大学联盟创建的博士后学员专业发展 MOOC 的方法。大多数评估 MOOC 的方法侧重于通过在线内容或点击流数据分析参与度、课程分配的结果、自我报告的学习结果、课程完成和用户路径。很少有已发表的评估详细描述在线课程中的学习方式以及参与者预期的中长期认知和行为结果。这项工作旨在通过应用变革理论来指导那些设计、实施和评估 MOOC 的人，将评估重点放在学习过程上。这种方法可以作为传统的 MOOC 评估方法的补充。我们描述了我们如何与 MOOC 团队合作构建内容，为 MOOC 中的每个模块（或课程）制定变革理论，并使用变革理论来指导对短期和中期参与者认知、情感和态度的评估，以及行为结果。最后，我们分享了在 MOOC 开发的设计和评估阶段实施变革理论的经验教训和建议。情感和态度以及行为结果。最后，我们分享了在 MOOC 开发的设计和评估阶段实施变革理论的经验教训和建议。情感和态度以及行为结果。最后，我们分享了在 MOOC 开发的设计和评估阶段实施变革理论的经验教训和建议。