Research Article
Year 2020,
Volume: 7 Issue: 2, 145 - 158, 13.06.2020
Abstract
This study aims to conduct differential item functioning analyses in the context of cognitive diagnosis assessments using various formulations of the Wald test. In implementing the Wald test, two scenarios are considered: one where the underlying reduced model can be assumed; and another where a saturated CDM is used. Illustration of the different Wald test to detect DIF in CDM data was based on the items’ performance of the Proportional Reasoning test among low- and high-performing school students. A benchmark simulation study was included to compare the performance of the Wald test in each scenario. The agreement of the latent attribute classification based on different cognitive diagnosis models was also discussed.
References
- Camilli, G. (2006). Test fairness. Educational Measurement, 4, 221-256.
- Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.
- de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179-199.
- de la Torre, J., & Douglas, J. A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333-353.
- de la Torre, J., & Lee, Y. S. (2013). Evaluating the wald test for item-level comparison of saturated and reduced models in cognitive diagnosis. Journal of Educational Measurement, 50, 355-373.
- Doornik, J. A. (2002). An object-oriented matrix programming using Ox (Version 3.1) [Computer software]. London, UK: Timberlake Consultants Press.
- Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 301-321.
- Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (p. 129–145). Hilldale, NJ: Lawrence Earlbaum Associates.
- Hou, L., de la Torre, J., & Nandakumar, R. (2014). Differential item functioning assessment in cognitive diagnostic modeling: Application of the wald test to investigate DIF in the DINA model. Journal of Educational Measurement, 51, 98-125.
- Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258-272.
- Li, F. (2008). A modified higher-order DINA model for detecting differential item functioning and differential attribute functioning (Doctoral dissertation). University of Georgia, Athens, GA.
- Ma, W., Iaconangelo, C., & de la Torre, J. (2016). Model similarity, model selection, and attribute classification. Applied Psychological Measurement, 40, 200-217.
- Milewski, G. B., & Baron, P. A. (2002). Extending DIF methods to inform aggregate reports on cognitive skills. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, LA.
- Morrison, D. F. (1967). Multivariate statistical methods. New York, NY: McGraw-Hill.
- Shealy, R., & Stout, W. (1993). A model-based standardization approach that separates true bias/dif from group ability differences and detects test bias/dtf as well as item bias/dif. Psychometrika, 58, 159-194.
- Tatsuoka, K. K. (1984). Analysis of errors in fraction addition and subtraction problems. Computer-based Education Research Laboratory, University of Illinois.
- Terzi, R. (2017). New Q-matrix validation procedures (Doctoral dissertation). Rutgers, The State University of New Jersey, New Brunswick, NJ.
- Tjoe, H., & de la Torre, J. (2014). The identification and validation process of proportional reasoning attributes: An application of a cognitive diagnosis modeling framework. Mathematics Education Research Journal, 26, 237-255.
- Zhang, W. (2006). Detecting differential item functioning using the DINA model (Doctoral dissertation). University of North Carolina at Greensboro, Greensboro, NC.
- Zumbo, B. D. (2007). Three generations of dif analyses: Considering where it has been where it is now, and where it is going. Language Assessment Quarterly, 4, 223-233.
Year 2020,
Volume: 7 Issue: 2, 145 - 158, 13.06.2020
Abstract
This study aims to conduct differential item functioning analyses in the context of cognitive diagnosis assessments using various formulations of the Wald test. In implementing the Wald test, two scenarios are considered: one where the underlying reduced model can be assumed; and another where a saturated CDM is used. Illustration of the different Wald test to detect DIF in CDM data was based on the items’ performance of the Proportional Reasoning test among low- and high-performing school students. A benchmark simulation study was included to compare the performance of the Wald test in each scenario. The agreement of the latent attribute classification based on different cognitive diagnosis models was also discussed.
References
- Camilli, G. (2006). Test fairness. Educational Measurement, 4, 221-256.
- Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.
- de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179-199.
- de la Torre, J., & Douglas, J. A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333-353.
- de la Torre, J., & Lee, Y. S. (2013). Evaluating the wald test for item-level comparison of saturated and reduced models in cognitive diagnosis. Journal of Educational Measurement, 50, 355-373.
- Doornik, J. A. (2002). An object-oriented matrix programming using Ox (Version 3.1) [Computer software]. London, UK: Timberlake Consultants Press.
- Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 301-321.
- Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (p. 129–145). Hilldale, NJ: Lawrence Earlbaum Associates.
- Hou, L., de la Torre, J., & Nandakumar, R. (2014). Differential item functioning assessment in cognitive diagnostic modeling: Application of the wald test to investigate DIF in the DINA model. Journal of Educational Measurement, 51, 98-125.
- Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258-272.
- Li, F. (2008). A modified higher-order DINA model for detecting differential item functioning and differential attribute functioning (Doctoral dissertation). University of Georgia, Athens, GA.
- Ma, W., Iaconangelo, C., & de la Torre, J. (2016). Model similarity, model selection, and attribute classification. Applied Psychological Measurement, 40, 200-217.
- Milewski, G. B., & Baron, P. A. (2002). Extending DIF methods to inform aggregate reports on cognitive skills. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, LA.
- Morrison, D. F. (1967). Multivariate statistical methods. New York, NY: McGraw-Hill.
- Shealy, R., & Stout, W. (1993). A model-based standardization approach that separates true bias/dif from group ability differences and detects test bias/dtf as well as item bias/dif. Psychometrika, 58, 159-194.
- Tatsuoka, K. K. (1984). Analysis of errors in fraction addition and subtraction problems. Computer-based Education Research Laboratory, University of Illinois.
- Terzi, R. (2017). New Q-matrix validation procedures (Doctoral dissertation). Rutgers, The State University of New Jersey, New Brunswick, NJ.
- Tjoe, H., & de la Torre, J. (2014). The identification and validation process of proportional reasoning attributes: An application of a cognitive diagnosis modeling framework. Mathematics Education Research Journal, 26, 237-255.
- Zhang, W. (2006). Detecting differential item functioning using the DINA model (Doctoral dissertation). University of North Carolina at Greensboro, Greensboro, NC.
- Zumbo, B. D. (2007). Three generations of dif analyses: Considering where it has been where it is now, and where it is going. Language Assessment Quarterly, 4, 223-233.
There are 20 citations in total.
Details
| Primary Language | English |
|---|---|
| Subjects | Studies on Education |
| Journal Section | Articles |
| Authors | |
| Publication Date | June 13, 2020 |
| Submission Date | February 16, 2020 |
| Published in Issue | Year 2020 Volume: 7 Issue: 2 |
Cited By
Detecting Differential Item Functioning Using Multiple-Group Cognitive Diagnosis Models
Applied Psychological Measurement
https://doi.org/10.1177/0146621620965745
Comparison of Methods Used in Detection of DIF in Cognitive Diagnostic Models with Traditional Methods: Applications in TIMSS 2011
Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi
https://doi.org/10.21031/epod.1218144