Abstract
Saturated diagnostic classification models (DCM) can flexibly accommodate various relationships among attributes to diagnose individual attribute mastery, and include various important DCMs as sub-models. However, the existing formulations of the saturated DCM are not better suited for deriving conditionally conjugate priors of model parameters. Because their derivation is the key in developing a variational Bayes (VB) inference algorithm, in the present study, we proposed a novel mixture formulation of saturated DCM. Based on it, we developed a VB inference algorithm of the saturated DCM that enables us to perform scalable and computationally efficient Bayesian estimation. The simulation study indicated that the proposed algorithm could recover the parameters in various conditions. It has also been demonstrated that the proposed approach is particularly suited to the case when new data become sequentially available over time, such as in computerized diagnostic testing. In addition, a real educational dataset was comparatively analyzed with the proposed VB and Markov chain Monte Carlo (MCMC) algorithms. The result demonstrated that very similar estimates were obtained between the two methods and that the proposed VB inference was much faster than MCMC. The proposed method can be a practical solution to the problem of computational load.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Beal, M. J. (2003). Variational algorithms for approximate Bayesian inference. [Doctoral dissertation, University College London]. https://www.cse.buffalo.edu/faculty/mbeal/thesis/.
Bishop, M. (2006). Pattern recognition and machine learning. Pattern Recognition. https://doi.org/10.1641/B580519.
Blei, D. M., Kucukelbir, A., & McAuliffe, J. D. (2017). Variational inference: A review for statisticians. Journal of the American Statistical Association, 112, 859–877. https://doi.org/10.1080/01621459.2017.1285773.
Brooks, S., Gelman, A., Jones, G. L., & Meng, X.-L. (2011). Handbook of Markov chain Monte Carlo. Roken Sound Parkway: CRC Press.
Chen, J., & de la Torre, J. (2014). A procedure for diagnostically modeling extant large-scale assessment data? The case of the programme for international student assessment in reading. Psychology, 5, 1967–1978. https://doi.org/10.4236/psych.2014.518200.
Chen, J., de la Torre, J., & Zhang, Z. (2013). Relative and absolute fit evaluation in cognitive diagnosis modeling. Journal of Educational Measurement, 50(2), 123–140. https://doi.org/10.1111/j.1745-3984.2012.00185.x.
Chen, Y., Culpepper, S. A., Chen, Y., & Douglas, J. (2018). Bayesian estimation of the DINA Q matrix. Psychometrika, 83(1), 89–108. https://doi.org/10.1007/s11336-017-9579-4.
Chiu, C. Y., & Douglas, J. (2013). A nonparametric approach to cognitive diagnosis by proximity to ideal response patterns. Journal of Classification, 30, 225–250. https://doi.org/10.1007/s00357-013-9132-9.
Culpepper, S. A. (2015). Bayesian estimation of the DINA model with Gibbs sampling. Journal of Educational and Behavioral Statistics, 40, 454–476. https://doi.org/10.3102/1076998615595403.
Culpepper, S. A. (2019). Estimating the cognitive diagnosis Q matrix with expert knowledge: Application to the fraction-subtraction dataset. Psychometrika, 84, 333–357. https://doi.org/10.1007/s11336-018-9643-8.
Culpepper, S. A., & Hudson, A. (2018). An improved strategy for Bayesian estimation of the reduced reparameterized unified model. Applied Psychological Measurement, 42(2), 99–115. https://doi.org/10.1177/0146621617707511.
de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179–199. https://doi.org/10.1007/S11336-011-9207-7.
Dempster, A. P., Laird, N. M., & Rubin, D. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 39, 1–38. https://doi.org/10.2307/2984875.
Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–472. https://doi.org/10.1214/ss/1177011136.
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian data analysis (3rd ed.). Chapman and Hall/CRC .
George, A. C., Robitzsch, A., Kiefer, T., Groß, J., & Ünlü, A. (2016). The R package CDM for cognitive diagnosis models. Journal of Statistical Software. https://doi.org/10.18637/jss.v074.i02.
Grimmer, J. (2011). An introduction to Bayesian inference via variational approximations. Political Analysis, 19, 32–47. https://doi.org/10.1093/pan/mpq027.
Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 301–321. https://doi.org/10.1111/j.1745-3984.1989.tb00336.x.
Hartz, S., & Roussos, L. (2008). The fusion model for skills diagnosis: Blending theory with practice. ETS Research Report Series, 08–71, 1–57. Retrieved from https://www.ets.org/Media/Research/pdf/RR-08-71.pdf.
Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210. https://doi.org/10.1007/S11336-008.
Jeon, M., Rijmen, F., & Rabe-hesketh, S. (2017). A variational maximization–maximization algorithm for generalized linear mixed models with crossed random effects. Psychometrika, 82, 693–716. https://doi.org/10.1007/s11336-017-9555-z.
Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272. https://doi.org/10.1177/01466210122032064.
Lee, M. D., & Wagenmakers, E.-J. (2013). Bayesian cognitive modeling: A practical course. Cambridge: Cambridge University Press.
Lee, Y.-S., Park, Y. S., & Taylan, D. (2011). A cognitive diagnostic modeling of attribute mastery in massachusetts, minnesota, and the U.S. national sample using the TIMSS 2007. International Journal of Testing, 11, 144–177. https://doi.org/10.1080/15305058.2010.534571.
Leighton, J. P., & Gierl, M. J. (Eds.). (2007). Cognitive diagnostic assessment for education: Theory and applications. New York: Cambridge University Press.
Li, H., Hunter, C. V., & Lei, P.-W. (2016). The selection of cognitive diagnostic models for a reading comprehension test. Language Testing, 33(2012), 1–35. https://doi.org/10.1177/0265532215590848.
Liu, J., Xu, G., & Ying, Z. (2012). Data-driven learning of Q-matrix. Applied Psychological Measurement, 36, 548–564. https://doi.org/10.1177/0146621612456591.
Ma, W., & de la Torre, J. (2020). GDINA: An R package for cognitive diagnosis modeling. Journal of Statistical Software, 93(10), 1–26. https://doi.org/10.18637/jss.v093.i14.
Macready, G. B., & Dayton, C. M. (1977). The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics, 2, 99–120.
Nakajima, S., Watanabe, K., & Sugiyama, M. (2019). Variational Bayesian learning theory. New York: Cambridge University Press. https://doi.org/10.1017/9781139879354.
Plummer, M. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. In The 3rd international workshop on distributed statistical computing, 124, 1–8. Retrieved from http://www.ci.tuwien.ac.at/Conferences/DSC-2003/.
R Core Team (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/.
Rijmen, F., Jeon, M., & Rabe-Hesketh, S. (2016). Variational approximation methods. In W. J. van der Linden (Ed.), Handbook of item response theory, volume two: Statistical tools (pp. 259–270). Boca Raton: CRC Press.
Rupp, A. A., Templin, J. L., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods and applications. New York: Guilford Press.
Sessoms, J., & Henson, R. A. (2018). Applications of diagnostic classification models: A literature review and critical commentary. Measurement: Interdisciplinary Research and Perspectives, 16, 1–17. https://doi.org/10.1080/15366367.2018.1435104.
Tatsuoka, K. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Jounal of Educational Measurement, 20, 345–354. https://doi.org/10.1111/j.1745-3984.1983.tb00212.x.
Tatsuoka, K. K., Corter, J. E., & Tatsuoka, C. (2004). Patterns of diagnosed mathematical content and process skills in TIMSS-R across a sample of 20 countries. American Educational Research Journal, 41, 901–926. https://doi.org/10.3102/00028312041004901.
Templin, J., & Bradshaw, L. (2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies. Psychometrika, 79, 317–339. https://doi.org/10.1007/s11336-013-9362-0.
Templin, J., & Hoffman, L. (2013). Obtaining diagnostic classification model estimates using Mplus. Educational Measurement: Issues and Practice, 32, 37–50. https://doi.org/10.1111/emip.12010.
Templin, J. L., & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287–305. https://doi.org/10.1037/1082-989X.11.3.287.
Tzikas, D. G., Likas, A. C., & Galatsanos, N. P. (2008). The variational approximation for Bayesian inference. IEEE Signal Processing Magazine. https://doi.org/10.1109/MSP.2008.929620.
van Ravenzwaaij, D., Cassey, P., & Brown, S. D. (2018). A simple introduction to Markov Chain Monte-Carlo sampling. Psychonomic Bulletin and Review, 25(eq1), 143–154. https://doi.org/10.3758/s13423-016-1015-8.
von Davier, M. (2008). A general diagnostic model applied to language testing data. The British Journal of Mathematical and Statistical Psychology, 61(Pt 2), 287–307. https://doi.org/10.1348/000711007X193957.
von Davier, M. (2010). Hierarchical mixtures of diagnostic models. Psychological Test and Assessment Modeling, 52, 8–28.
von Davier, M. (2014). The log-linear cognitive diagnostic model (LCDM) as a special case of the general diagnostic model (GDM). ETS Research Report Series (Vol. RR–14-40). Princeton, NJ. https://doi.org/10.1002/ets2.12043.
Wang, W.-C., & Qiu, X.-L. (2019). Multilevel modeling of cognitive diagnostic assessment: The multilevel DINA model example. Applied Psychological Measurement, 43, 34–50. https://doi.org/10.1177/0146621618765713.
Wagenmakers, E.-J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J., et al. (2018). Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications. Psychonomic Bulletin & Review, 25, 35–57. https://doi.org/10.3758/s13423-017-1343-3.
White, A., & Murphy, T. B. (2014). BayesLCA? An R package for Bayesian latent class. Journal of Statistical Software, 61, 1–28. https://doi.org/10.1080/01621459.2016.1231612.
Yamaguchi, K. (2020). Variational Bayesian inference for the multiple-choice DINA model. Behaviormetrika, 47, 159–187. https://doi.org/10.1007/s41237-020-00104-w.
Yamaguchi, K., & Okada, K. (2018). Comparison among cognitive diagnostic models for the TIMSS 2007 fourth grade mathematics assessment. PLoS ONE, 13, e0188691. https://doi.org/10.1371/journal.pone.0188691.
Yamaguchi, K., & Okada, K. (2020). Variational Bayes inference for the DINA Model. Journal of Educational and Behavioral Statistics, 45(5), 569–597. https://doi.org/10.3102/1076998620911934.
Zhan, P., Jiao, H., Man, K., & Wang, L. (2019). Using JAGS for Bayesian cognitive diagnosis modeling: A tutorial. Journal of Educational and Behavioral Statistics, 44, 473–503. https://doi.org/10.3102/1076998619826040.
Zhan, P., Liao, M., & Bian, Y. (2018). Joint testlet cognitive diagnosis modeling for paired local item dependence in response times and response accuracy. Frontiers in Psychology, 9(APR), 1–14. https://doi.org/10.3389/fpsyg.2018.00607.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
We have no conflict of interest to declare.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work was supported by JSPS Grant-in-Aid for JSPS Research Fellow 18J01312, JSPS KAKANHI 17H04787, and JSPS KAKENHI 20H01720.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Yamaguchi, K., Okada, K. Variational Bayes Inference Algorithm for the Saturated Diagnostic Classification Model. Psychometrika 85, 973–995 (2020). https://doi.org/10.1007/s11336-020-09739-w
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-020-09739-w