Abstract
This paper presents a dynamic tree-based item response (IRTree) model as a novel extension of the autoregressive generalized linear mixed effect model (dynamic GLMM). We illustrate the unique utility of the dynamic IRTree model in its capability of modeling differentiated processes indicated by intensive polytomous time-series eye-tracking data. The dynamic IRTree was inspired by but is distinct from the dynamic GLMM which was previously presented by Cho, Brown-Schmidt, and Lee (Psychometrika 83(3):751–771, 2018). Unlike the dynamic IRTree, the dynamic GLMM is suitable for modeling intensive binary time-series eye-tracking data to identify visual attention to a single interest area over all other possible fixation locations. The dynamic IRTree model is a general modeling framework which can be used to model change processes (trend and autocorrelation) and which allows for decomposing data into various sources of heterogeneity. The dynamic IRTree model was illustrated using an experimental study that employed the visual-world eye-tracking technique. The results of a simulation study showed that parameter recovery of the model was satisfactory and that ignoring trend and autoregressive effects resulted in biased estimates of experimental condition effects in the same conditions found in the empirical study.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Missing at random is missingness given other observations which means that all the information about when missingness occurs is contained in the observed information.
As explained below, we used deviation coding (-1 vs. 1) in model fitting, which compares each level to the grand mean. The interpretations of the AR(1) effects are provided in the supplementary materials.
Here, a subscript i is for a set of items which can be identified by a trial id and a person id.
Across the entire 111 time points (controlling for the other covariates), the trend estimate of 0.006 at Node 1 means that there is 1.174 (\(=exp(0.161)\)) increase in odds ratio for the target and competitor fixation and the trend estimate of 0.031 indicates that there is 1.598 (\(=exp(0.469)\)) in odds ratio for the target fixation at Node 2.
When Model 2 from Table 4 is estimated with just one random person intercept for both nodes, model fit is poorer (AIC = 181,432, BIC = 181,627), compared to the full Model 2 in Table 2 which has two random person intercepts, one per node (AIC = 180,262, BIC = 180,554). The correlation between the two random person intercepts is even smaller for Model 1 than for Model 2 (.414 vs. .705).
References
Allopenna, P. D., Magnuson, J. S., & Tanenhaus, M. K. (1998). Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. Journal of Memory and Language, 38, 419–439. https://doi.org/10.1006/jmla.1997.2558.
Altmann, G. T., & Kamide, Y. (1999). Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition, 73, 247–264. https://doi.org/10.1016/S0010-0277(99)00059-1.
Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412. https://doi.org/10.1016/j.jml.2007.12.005.
Barr, D. J. (2008a). Analyzing ’visual world’ eyetracking data using multilevel logistic regression. Journal of Memory and Language, 59, 457–474. https://doi.org/10.1016/j.jml.2007.09.002.
Barr, D. J. (2008b). Pragmatic expectations and linguistic evidence: Listeners anticipate but do not integrate common ground. Cognition, 109, 18–40. https://doi.org/10.1016/j.cognition.2008.07.005.
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68, 255–278. https://doi.org/10.1016/j.jml.2012.11.001.
Barrett, H. C., Broesch, T., Scott, R. M., He, Z., Baillargeon, R., & Wu, D. (2013). Early false-belief understanding in traditional non-western societies. Proceedings of the Royal Society of London B: Biological Sciences, 280, 2012–2654. https://doi.org/10.1098/rspb.2012.2654.
Bartolucci, F., & Nigro, V. (2010). A dynamic model for binary panel data with unobserved heterogeneity admitting a \(n\)-consistent conditional estimator. Econometrica, 78, 719–733. https://doi.org/10.3982/ecta7531.
Batchelder, W. H., & Crowther, C. S. (1997). Multinomial processing tree models of factorial categorization. Journal of Mathematical Psychology, 41, 45–55. https://doi.org/10.1006/jmps.1997.1146.
Batchelder, W. H., & Riefer, D. M. (1999). Theoretical and empirical review of multinomial process tree modeling. Psychonomic Bulletin & Review, 6, 57–86. https://doi.org/10.3758/bf03210812.
Bates, D., Mächler, M., Bolker, B., Walker, S., Christensen, R. H. B., Singmann, H., et al. (2018). Package “lme4”: Linear mixed-effects models using ‘eigen’ and s4. Retrieved March 10, 2018 from https://cran.r-project.org/web/packages/lme4/lme4.pdf.
Böckenholt, U. (2012). Modeling multiple response processes in judgment and choice. Psychological Methods, 17(4), 665–678. https://doi.org/10.1037/a0028111.
Böckenholt, U. (2017). Measuring response styles in Likert items. Psychological Methods, 22, 69–83. https://doi.org/10.1037/met0000106.
Bringmann, L. F., Hamaker, E. L., Vigo, D. E., Aubert, A., Borsboom, D., & Tuerlinckx, F. (2017). Changing dynamics: Time-varying autoregressive models using generalized additive modeling. Psychological Methods, 22, 409–425. https://doi.org/10.1037/met0000085.
Brown-Schmidt, S. (2009a). The role of executive function in perspective-taking during on-line language comprehension. Psychonomic Bulletin & Review, 16, 893–900. https://doi.org/10.3758/pbr.16.5.893.
Brown-Schmidt, S. (2009b). Partner-specific interpretation of maintained referential precedents during interactive dialog. Journal of Memory and Language, 61, 171–190. https://doi.org/10.1016/j.jml.2009.04.003.
Brown-Schmidt, S. (2012). Beyond common and privileged: Gradient representations of common ground in real-time language use. Language and Cognitive Processes, 27, 62–89. https://doi.org/10.1080/01690965.2010.543363.
Brown-Schmidt, S., Gunlogson, C., & Tanenhaus, M. K. (2008). Addressees distinguish shared from private information when interpreting questions during interactive conversation. Cognition, 107, 1122–1134. https://doi.org/10.1016/j.cognition.2007.11.005.
Browne, M. W., & Nesselroade, J. R. (2005). Representing psychological processes with dynamic factor models: Some promising uses and extensions of autoregressive moving average time series models. In A. Maydeu-Olivares & J. J. McArdle (Eds.), Multivariate applications book series. Contemporary psychometrics: A festschrift for Roderick P. McDonald (pp. 415–452). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.
Burnham, K. P., & Anderson, D. R. (2004). Model selection and multimodel inference: A practical information-theoretical approach (2nd ed.). New York, NY: Springer. https://doi.org/10.1007/b97636.
Carpenter, B., Gelman, A., Hoffman, M., Lee, D., Goodrich, B., Betancourt, M., et al. (2017). Stan: A probabilistic programming language. Journal of Statistical Software, 76, 1–32. https://doi.org/10.18637/jss.v076.i01.
Chatfield, C. (2004). The analysis of time series: An introduction (6th ed.). London: Chapman Hall. https://doi.org/10.1007/978-1-4899-2921-1.
Cho, S.-J., & De Boeck, P. (2018). A note on \(N\) in Bayesian information criterion for item response models. Applied Psychological Measurement, 42, 169–172. https://doi.org/10.1177/0146621617726791.
Cho, S.-J., Brown-Schmidt, S., & Lee, W.-y. (2018). Autoregressive generalized linear mixed effect models with crossed random effects: An application to intensive binary time series eye-tracking data. Psychometrika, 83, 751–771. https://doi.org/10.1007/s11336-018-9604-2.
Clark, H. H. (1973). The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. Journal of Verbal Learning and Verbal Behavior, 12, 335–359. https://doi.org/10.1016/s0022-5371(73)80014-3.
Cox, M. D. (1970). The analysis of binary data. London: Chapman and Hall.
Craigmile, P. F., Peruggia, M., & Van Zandt, T. (2010). Detrending response time series. In S.-M. Chow, E. Ferrer, & F. Hsieh (Eds.), Statistical methods for modeling human dynamics: An interdisciplinary dialogue (pp. 213–240). Boca Raton, FL: Taylor & Francis.
Curran, P. J., & Bollen, K. A. (2001). The best of both worlds: Combining autoregressive and latent curve models. In L. M. Collins & A. G. Sayer (Eds.), Decade of behavior. New methods for the analysis of change. Washington, DC: American Psychological Association. https://doi.org/10.1037/10409-000.
Curran, P. J., Lee, T., Howard, A. L., Lane, S., & MacCallum, R. (2012). Disaggregating within-person and between-person effects in multilevel and structural equation growth models. In J. R. Harring & G. R. Hancock (Eds.), CILVR series on latent variable methodology. Advances in longitudinal methods in the social and behavioral sciences (pp. 217–253). Charlotte, NC: IAP Information Age Publishing.
De Boeck, P., & Partchev, I. (2012). IRTrees: Tree-based item response models of the GLMM family. Journal of Statistical Software, 48, 1–28. https://doi.org/10.18637/jss.v048.c01.
De Boeck, P., & Cho, S.-J. (2019). IRTree modeling of cognitive processes based on outcome and intermediate data. College Park: Maryland Assessment Research Center (MARC).
Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence-Erlbaum.
Fahrmeir, L. (1992). Posterior mode estimation by extended kalman filtering for multivariate dynamic generalized linear models. Journal of the American Statistical Association, 87, 501–509. https://doi.org/10.2307/2290283.
Gamerman, D. (1998). Markov chain Monte Carlo for dynamic generalised linear models. Biometrika, 85, 215–227. https://doi.org/10.1093/biomet/85.1.215.
Greven, S., & Kneib, T. (2010). On the behaviour of marginal and conditional AIC in linear mixed models. Biometrika, 97, 773–789. https://doi.org/10.1093/biomet/asq042.
Hallett, P. E. (1986). Eye movements. In K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.), Handbook of perception and human performance (Vol. 1, pp. 10.1–10.112). New York, NY: Wiley.
Hamaker, E. L., & Grasman, R. P. (2015). To center or not to center? Investigating inertia with a multilevel autoregressive model. Frontiers in Psychology, 5, 1492. https://doi.org/10.3389/fpsyg.2014.01492.
Hanna, J. E., Tanenhaus, M. K., & Trueswell, J. C. (2003). The effects of common ground and perspective on domains of referential interpretation. Journal of Memory and Language, 49, 43–61. https://doi.org/10.1016/s0749-596x(03)00022-6.
Hsiao, C. (2003). Analysis of panel data. New York: Cambridge University Press. https://doi.org/10.1017/CBO9781139839327.
Hung, Y., Zarnitsyna, V., Zhang, Y., Zhu, C., & Wu, C. J. (2008). Binary time series modeling with application to adhesion frequency experiments. Journal of the American Statistical Association, 103, 1248–1259. https://doi.org/10.1198/016214508000000508.
Jaeger, T. F. (2008). Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language, 59, 434–446. https://doi.org/10.1016/j.jml.2007.11.007.
Jahng, S., Wood, P. K., & Trull, T. J. (2008). Analysis of affective instability in ecological momentary assessment: Indices using successive difference and group comparison via multilevel modeling. Psychological Methods, 13, 354–75. https://doi.org/10.1037/a0014173.
Jeon, M., & De Boeck, P. (2016). A generalized item response tree model for psychological assessments. Behavior Research Methods, 48, 1070–1085. https://doi.org/10.3758/s13428-015-0631-y.
Jeon, M., & Rabe-Hesketh, S. (2016). An autoregressive growth model for longitudinal item analysis. Psychometrika, 81, 830–850. https://doi.org/10.1007/s11336-015-9489-2.
Klauer, K. C. (2010). Hierarchical multinomial processing tree models: A latent-trait approach. Psychometrika, 75, 70–98. https://doi.org/10.1007/s11336-009-9141-0.
Koval, P., Kuppens, P., Allen, N. B., & Sheeber, L. (2012). Getting stuck in depression: The roles of rumination and emotional inertia. Cognition and Emotion, 26, 1412–1427. https://doi.org/10.1080/02699931.2012.667392.
Kuppens, P., Allen, N. B., & Sheeber, L. B. (2010). Emotional inertia and psychological adjustment. Psychological Science, 21, 984–991. https://doi.org/10.1177/0956797610372634.
Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthén, B. (2008). The multilevel latent covariate model: A new, more reliable approach to group-level effects in contextual studies. Psychological Methods, 13, 203–229. https://doi.org/10.1037/a0012869.
MacDonald, M. C. (2013). How language production shapes language form and comprehension. Frontiers in Psychology, 4, 226. https://doi.org/10.3389/fpsyg.2013.00226.
Matzke, D., Dolan, C. V., Batchelder, W. H., & Wagenmakers, E.-J. (2015). Bayesian estimation of multinomial processing tree models with heterogeneity in participants and items. Psychometrika, 80, 205–235. https://doi.org/10.1007/s11336-013-9374-9.
McMurray, B., Samelson, V. M., Lee, S. H., & Tomblin, J. B. (2010). Individual differences in online spoken word recognition: Implications for SLI. Cognitive Psychology, 60, 1–39. https://doi.org/10.1016/j.cogpsych.2009.06.003.
McMurray, B., Klein-Packard, J., & Tomblin, J. B. (2019). A real-time mechanism underlying lexical deficits in developmental language disorder: Between-word inhibition. Cognition, 191, 104000. https://doi.org/10.1016/j.cognition.2019.06.012.
Merali, N., Veeramootoo, D., & Singh, S. (2019). Eye-tracking technology in surgical training. Journal of Investigative Surgery, 32, 587–593. https://doi.org/10.1080/08941939.2017.1404663.
Molenaar, P. C. (1985). A dynamic factor model for the analysis of multivariate time series. Psychometrika, 50, 181–202. https://doi.org/10.1007/bf02294246.
Mozuraitis, M., Chambers, C. G., & Daneman, M. (2015). Privileged versus shared knowledge about object identity in real-time referential processing. Cognition, 142, 148–165. https://doi.org/10.1016/j.cognition.2015.05.001.
Nadig, A. S., & Sedivy, J. C. (2002). Evidence of perspective-taking constraints in children’s on-line reference resolution. Psychological Science, 13(4), 329–336. https://doi.org/10.1111/1467-9280.00460.
Nickell, S. (1981). Biases in dynamic models with fixed effects. Econometrica: Journal of the Econometric Society, 49, 1417–1426. https://doi.org/10.2307/1911408.
Qu, S., & Chai, J. Y. (2007). An exploration of eye gaze in spoken language processing for multimodal conversational interfaces. In C. L. Sidner, T. Schultz, M. Stone, & C. Zhai (Eds.), NAACL HLT 2007-human language technologies 2007: The conference of the North American chapter of the association for computational linguistics, proceedings of the main conference (pp. 284–291). Rochester, NY: The Association for Computational Linguistics.
Quené, H., & van den Bergh, H. (2008). Examples of mixed-effects modeling with crossed random effects and with binomial data. Journal of Memory and Language, 59, 413–425. https://doi.org/10.1016/j.jml.2008.02.002.
R Core Team. (2017). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. Retrieved April 5, 2017 from http://www.R-project.org/.
Richardson, D. C., & Spivey, M. J. (2004). Eye tracking: Characteristics and methods. In G. Wnek & G. Bowlin (Eds.), Encyclopedia of biomaterials and biomedical engineering (pp. 1028–1042). Boca Raton: CRC Press.
Riefer, D. M., & Batchelder, W. H. (1988). Multinomial modeling and the measurement of cognitive processes. Psychological Review, 95, 318–339. https://doi.org/10.1037//0033-295x.95.3.318.
Rubin, D. B. (1976). Inference and missing data. Biometrika, 63, 581–592. https://doi.org/10.2307/2335739.
Rubin, R. D., Brown-Schmidt, S., Duff, M. C., Tranel, D., & Cohen, N. J. (2011). How do I remember that I know you know that I know? Psychological Science, 22, 1574–1582. https://doi.org/10.1177/0956797611418245.
Ryskin, R. A., Benjamin, A. S., Tullis, J., & Brown-Schmidt, S. (2015). Perspective-taking in comprehension, production, and memory: An individual differences approach. Journal of Experimental Psychology: General, 144, 898–915. https://doi.org/10.1037/xge0000093.
Salverda, A. P., Kleinschmidt, D., & Tanenhaus, M. K. (2014). Immediate effects of anticipatory coarticulation in spoken-word recognition. Journal of Memory and Language, 71, 145–163. https://doi.org/10.1016/j.jml.2013.11.002.
Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461–464. https://doi.org/10.1214/aos/1176344136.
Sedivy, J. C. (2007). Implicature during real time conversation: A view from language processing research. Philosophy Compass, 2, 475–496. https://doi.org/10.1111/j.1747-9991.2007.00082.x.
StataCorp. (2017). Stata statistical software: Release 15. College Station, TX: StataCorp LLC.
Staudte, M., & Crocker, M. W. (2011). Investigating joint attention mechanisms through spoken human–robot interaction. Cognition, 120, 268–291. https://doi.org/10.1016/j.cognition.2011.05.005.
Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268, 1632–1634. https://doi.org/10.1126/science.7777863.
Tutz, G. (1990). Sequential item response models with an ordered response. British Journal of Mathematical and Statistical Psychology, 43, 39–55. https://doi.org/10.1111/j.2044-8317.1990.tb00925.x.
Walker, G. M., Hickok, G., & Fridriksson, J. (2018). A cognitive psychometric model for assessment of picture naming abilities in aphasia. Psychological Assessment, 30, 809–826. https://doi.org/10.1037/pas0000529.
Wang, L. P., Hamaker, E., & Bergeman, C. S. (2012). Investigating inter-individual differences in short-term intra-individual variability. Psychological Methods, 17, 567–581. https://doi.org/10.1037/a0029317.
West, M., Harrison, P. J., & Migon, H. S. (1985). Dynamic generalized linear models and Bayesian forecasting. Journal of the American Statistical Association, 80, 73–83. https://doi.org/10.2307/2288042.
Acknowledgements
We thank an associate editor and five anonymous reviewers for their helpful comments.
Funding
Funding was provided in part by the National Science Foundation (SES 1851690) to Sun-Joo Cho, Sarah Brown-Schmidt, and Paul De Boeck. The original data collection was supported in part by the National Science Foundation (BCS 1257029 and BCS 1556700) to Sarah Brown-Schmidt. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Cho, SJ., Brown-Schmidt, S., Boeck, P.D. et al. Modeling Intensive Polytomous Time-Series Eye-Tracking Data: A Dynamic Tree-Based Item Response Model. Psychometrika 85, 154–184 (2020). https://doi.org/10.1007/s11336-020-09694-6
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-020-09694-6