Given the psychometric limitations associated with many of the existing critical thinking assessments as well as the lack of these assessments utilising constructed-response items, there is a need for critical thinking assessments that are not only psychometrically sound but also allow test-takers to articulate their reasoning. Compared to selected-response items, in which test-takers are required to select the ‘correct’ answer from a list of alternatives, constructed-response items require test-takers to construct, or generate, their answer. This study had two aims: (1) evaluate the reliability and validity of a pilot critical thinking assessment tool that predominantly consists of constructed-response items, and (2) conduct an exploratory content analysis of higher education students’ responses, including a comparison between participants’ selected and constructed responses. A total of 95 undergraduate students were included in the study, with participants recruited from psychology, chiropractic and computer science programs. This study found preliminary evidence of inter-rater reliability for the constructed-response portion of the assessment tool yet mixed evidence regarding the validity of the tool. This study also found several discrepancies between participants’ selected and constructed responses, therefore raising concerns regarding the use of selected-response items in critical thinking assessments. These findings have important implications for the assessment of higher education students’ critical thinking skills based on the quality of their reasoning rather than merely their ability to ‘select’ the correct answer.

鉴于与许多现有批判性思维评估相关的心理测量限制以及这些评估缺乏利用构建反应项目，因此需要批判性思维评估不仅在心理测量上合理，而且还允许应试者阐明他们的观点。推理。与要求考生从备选方案列表中选择“正确”答案的选定回答项目相比，构建回答项目要求考生构建或生成他们的答案。本研究有两个目的：(1) 评估主要由构建反应项目组成的试点批判性思维评估工具的可靠性和有效性，以及 (2) 对高等教育学生的反应进行探索性内容分析，包括参与者选择的和构建的响应之间的比较。该研究共包括 95 名本科生，参与者来自心理学、脊椎按摩疗法和计算机科学课程。这项研究发现了评估工具的构建响应部分的评估者间可靠性的初步证据，但关于该工具有效性的混合证据。这项研究还发现参与者的选择和构建的反应之间存在一些差异，因此引发了对在批判性思维评估中使用选择反应项目的担忧。这些发现对于基于推理质量而非仅仅“选择”正确答案的能力来评估高等教育学生的批判性思维技能具有重要意义。