Physics education research (PER) has long used concept inventories to investigate student learning over time and to compare performance across various student subpopulations. PER has traditionally used normalized gain to explore these questions but has begun to use established methods from other fields, including Cohen’s $d$, multiple linear regression, and linear mixed effects models. The choice of analysis method for examining student learning gains in PER is a current subject of debate. We synthesize this debate here by focusing on the research questions and interpretations that can be drawn using various statistical tools. We focus particularly on how results can be interpreted through an equity lens and what questions can and cannot be answered using different methods. Our results demonstrate the importance of clearly defining a research question and using appropriate tools to answer that question.

• Received 6 May 2020
• Accepted 18 June 2020

DOI:https://doi.org/10.1103/PhysRevPhysEducRes.16.010141

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society