Computerized classification testing (CCT) commonly chooses items maximizing information at the cut score, which yields the most information for decision-making. However, a corollary problem is that all examinees will be given the same set of items, resulting in high test overlap rate and unbalanced item bank usage, which threatens test security. Moreover, another pivotal issue for CCT is time control

Problem-solving strategies, defined as actions people select intentionally to achieve desired objectives, are distinguished from skills that are implemented unintentionally. In education, strategy-oriented instructions that guide students to form problem-solving strategies are found to be more effective for low-achieving students than the skill-oriented instructions designed for enhancing their skill

Most item response theory (IRT) models for dichotomous responses are based on probit or logit link functions which assume a symmetric relationship between the probability of a correct response and the latent traits of individuals taking a test. This assumption restricts the use of those models to the case in which all items behave symmetrically. On the other hand, asymmetric models proposed in the

Careless and insufficient effort responding (C/IER) on self-report measures results in responses that do not reflect the trait to be measured, thereby posing a major threat to the quality of survey data. Reliable approaches for detecting C/IER aid in increasing the validity of inferences being made from survey data. First, once detected, C/IER can be taken into account in data analysis. Second, approaches

Several psychometric tests and self-reports generate count data (e.g., divergent thinking tasks). The most prominent count data item response theory model, the Rasch Poisson Counts Model (RPCM), is limited in applicability by two restrictive assumptions: equal item discriminations and equidispersion (conditional mean equal to conditional variance). Violations of these assumptions lead to impaired reliability

Several psychometric tests and self-reports generate count data (e.g., divergent thinking tasks). The most prominent count data item response theory model, the Rasch Poisson Counts Model (RPCM), is limited in applicability by two restrictive assumptions: equal item discriminations and equidispersion (conditional mean equal to conditional variance). Violations of these assumptions lead to impaired reliability

Recently, the Urnings algorithm (Bolsinova et al., 2022, J. R. Stat. Soc. Ser. C Appl. Statistics, 71, 91) has been proposed that allows for tracking the development of abilities of the learners and the difficulties of the items in adaptive learning systems. It is a simple and scalable algorithm which is suited for large-scale applications in which large streams of data are coming into the system and

A family of score-based tests has been proposed in recent years for assessing the invariance of model parameters in several models of item response theory (IRT). These tests were originally developed in a maximum likelihood framework. This study discusses analogous tests for Bayesian maximum-a-posteriori estimates and multiple-group IRT models. We propose two families of statistical tests, which are

A family of score-based tests has been proposed in recent years for assessing the invariance of model parameters in several models of item response theory (IRT). These tests were originally developed in a maximum likelihood framework. This study discusses analogous tests for Bayesian maximum-a-posteriori estimates and multiple-group IRT models. We propose two families of statistical tests, which are

The g-and-h family of distributions is a computationally efficient, flexible option to model and simulate non-normal data. In spite of its popularity, there are several theoretical aspects of these distributions that need special consideration when they are used. In this paper some of these aspects are explored. In particular, through mathematical analysis it is shown that a popular multivariate generalization

Cognitive diagnosis models have become popular in educational assessment and are used to provide more individualized feedback about a student's specific strengths and weaknesses than traditional total scores. However, if the testing data are contaminated by certain biases or aberrant response patterns, such predictions may not be accurate. The current research objective is to develop a new person-fit

Cognitive diagnosis models have become popular in educational assessment and are used to provide more individualized feedback about a student's specific strengths and weaknesses than traditional total scores. However, if the testing data are contaminated by certain biases or aberrant response patterns, such predictions may not be accurate. The current research objective is to develop a new person-fit

A cluster randomized controlled trial (C-RCT) is common in educational intervention studies. Multilevel modelling (MLM) is a dominant analytic method to evaluate treatment effects in a C-RCT. In most MLM applications intended to detect an interaction effect, a single interaction effect (called a conflated effect) is considered instead of level-specific interaction effects in a multilevel design (called

A cluster randomized controlled trial (C-RCT) is common in educational intervention studies. Multilevel modelling (MLM) is a dominant analytic method to evaluate treatment effects in a C-RCT. In most MLM applications intended to detect an interaction effect, a single interaction effect (called a conflated effect) is considered instead of level-specific interaction effects in a multilevel design (called

In cognitive diagnostic assessment a property of the Q-matrix, usually referred to as completeness, warrants that the cognitive attributes underlying the observed behaviour can be uniquely assessed. Characterizations of completeness were first derived under the assumption of independent attributes, and are currently under investigation for interdependent attributes. The dominant approach considers

In cognitive diagnostic assessment a property of the Q-matrix, usually referred to as completeness, warrants that the cognitive attributes underlying the observed behaviour can be uniquely assessed. Characterizations of completeness were first derived under the assumption of independent attributes, and are currently under investigation for interdependent attributes. The dominant approach considers

We propose a new metric for evaluating the informativeness of a set of ratings from a single rater on a given scale. Such evaluations are of interest when raters rate numerous comparable items on the same scale, as occurs in hiring, college admissions, and peer review. Our exposition takes the context of peer review, which involves univariate and multivariate cardinal ratings. We draw on this context

Reliability of scores from psychological or educational assessments provides important information regarding the precision of measurement. The reliability of scores is however population dependent and may vary across groups. In item response theory, this population dependence can be attributed to differential item functioning or to differences in the latent distributions between groups and needs to

Interval estimation is one of the most frequently used methods in statistical science, employed to provide a range of credible values a parameter is located in after taking into account the uncertainty in the data. However, while this interpretation only holds for Bayesian interval estimates, these suffer from two problems. First, Bayesian interval estimates can include values which have not been corroborated

Reliability of scores from psychological or educational assessments provides important information regarding the precision of measurement. The reliability of scores is however population dependent and may vary across groups. In item response theory, this population dependence can be attributed to differential item functioning or to differences in the latent distributions between groups and needs to

Cochran's Q statistic is routinely used for testing heterogeneity in meta-analysis. Its expected value is also used in several popular estimators of the between-study variance, τ 2 . Those applications generally have not considered the implications of its use of estimated variances in the inverse-variance weights. Importantly, those weights make approximating the distribution of Q (more explicitly

Logistic regression models are a powerful research tool for the analysis of cross-classified data in which a categorical response variable is involved. In a logistic model, the effect of a covariate refers to odds, and the simple relationship between the coefficients and the odds ratio often makes these the parameters of interest due to their easy interpretation. In this article we present a distance-based

The Plackett-Luce model (PL) for ranked data assumes the forward order of the ranking process. This hypothesis postulates that the ranking process of the items is carried out by sequentially assigning the positions from the top (most liked) to the bottom (least liked) alternative. This assumption has been recently relaxed with the Extended Plackett-Luce model (EPL) through the introduction of the discrete

The aim of latent variable selection in multidimensional item response theory (MIRT) models is to identify latent traits probed by test items of a multidimensional test. In this paper the expectation model selection (EMS) algorithm proposed by Jiang et al. (2015) is applied to minimize the Bayesian information criterion (BIC) for latent variable selection in MIRT models with a known number of latent

Fleishman's power method is frequently used to simulate non-normal data with a desired skewness and kurtosis. Fleishman's method requires solving a system of nonlinear equations to find the third-order polynomial weights that transform a standard normal variable into a non-normal variable with desired moments. Most users of the power method seem unaware that Fleishman's equations have multiple solutions

A standard approach to distinguishing people’s risk preferences is to estimate a random utility model using a power utility function to characterize the preferences and a logit function to capture choice consistency. We demonstrate that with often-used choice situations, this model suffers from empirical underidentification, meaning that parameters cannot be estimated precisely. With simulations of

Approximately counting and sampling knowledge states from a knowledge space is a problem that is of interest for both applied and theoretical reasons. However, many knowledge spaces used in practice are far too large for standard statistical counting and estimation techniques to be useful. Thus, in this work we use an alternative technique for counting and sampling knowledge states from a knowledge

Approximately counting and sampling knowledge states from a knowledge space is a problem that is of interest for both applied and theoretical reasons. However, many knowledge spaces used in practice are far too large for standard statistical counting and estimation techniques to be useful. Thus, in this work we use an alternative technique for counting and sampling knowledge states from a knowledge

The result of a meta-analysis is conventionally pictured in the forest plot as a diamond, whose length is the 95% confidence interval (CI) for the summary measure of interest. The Diamond Ratio (DR) is the ratio of the length of the diamond given by a random effects meta-analysis to that given by a fixed effect meta-analysis. The DR is a simple visual indicator of the amount of change caused by moving

This article examines the Fisher information functions, , and explores implications for scoring of binary ideal point item response models. These models typically appear to have that are bimodal and identically equal to 0 at the ideal point. The article shows that this is an inherent property of ideal point IRT models, which either have this property or are indeterminate and thus violate the likelihood

Structural equation modelling (SEM) has evolved into two domains, factor-based and component-based, dependent on whether constructs are statistically represented as common factors or components. The two SEM domains are conceptually distinct, each assuming their own population models with either of the statistical construct proxies, and statistical SEM approaches should be used for estimating models

Random effects in longitudinal multilevel models represent individuals’ deviations from population means and are indicators of individual differences. Researchers are often interested in examining how these random effects predict outcome variables that vary across individuals. This can be done via a two-step approach in which empirical Bayes (EB) estimates of the random effects are extracted and then

Random effects in longitudinal multilevel models represent individuals' deviations from population means and are indicators of individual differences. Researchers are often interested in examining how these random effects predict outcome variables that vary across individuals. This can be done via a two-step approach in which empirical Bayes (EB) estimates of the random effects are extracted and then

Increasing use of innovative items in operational assessments has shedded new light on the polytomous testlet models. In this study, we examine performance of several scoring models when polytomous items exhibit random testlet effects. Four models are considered for investigation: the partial credit model (PCM), testlet-as-a-polytomous-item model (TPIM), random-effect testlet model (RTM), and fixed-effect

Among the various forms of response bias that can emerge with self-report rating scale assessments are those related to anchoring, the tendency for respondents to select categories in close proximity to the rating category used for the immediately preceding item. In this study we propose a psychometric model based on a multidimensional nominal model for response style that also simultaneously accommodates

In a previous paper, the evolution of certainty measured during a consensus-based small-group decision process was shown to oscillate to an equilibrium value for about two-thirds of the participants in the experiment. Starting from the observation that experimental participants are split into two groups, those for whom the evolution of certainty oscillates and those for whom it does not, in this paper

In this article we extend the framework of explanatory mixed IRT models to a more general class called explanatory additive IRT models. We do this by augmenting the linear predictors in terms of smooth functions. This development offers many new modeling options such as the inclusion of nonlinear covariate effects, the specification of various temporal and spatial dependency patterns, and parameter

The four-parameter logistic (4PL) item response model, which includes an upper asymptote for the correct response probability, has drawn increasing interest due to its suitability for many practical scenarios. This paper proposes a new Gibbs sampling algorithm for estimation of the multidimensional 4PL model based on an efficient data augmentation scheme (DAGS). With the introduction of three continuous

Consider a two-way ANOVA design. Generally, interactions are characterized by the difference between two measures of effect size. Typically the measure of effect size is based on the difference between measures of location, with the difference between means being the most common choice. This paper deals with extending extant results to two robust, heteroscedastic measures of effect size. The first

The effects of a treatment or an intervention on a count outcome are often of interest in applied research. When controlling for additional covariates, a negative binomial regression model is usually applied to estimate conditional expectations of the count outcome. The difference in conditional expectations under treatment and under control is then defined as the (conditional) treatment effect. While

Meta-analyses of correlation coefficients are an important technique to integrate results from many cross-sectional and longitudinal research designs. Uncertainty in pooled estimates is typically assessed with the help of confidence intervals, which can double as hypothesis tests for two-sided hypotheses about the underlying correlation. A standard approach to construct confidence intervals for the

Methods for the treatment of item non-response in attitudinal scales and in large-scale assessments under the pairwise likelihood (PL) estimation framework and under a missing at random (MAR) mechanism are proposed. Under a full information likelihood estimation framework and MAR, ignorability of the missing data mechanism does not lead to biased estimates. However, this is not the case for pseudo-likelihood

The non-response model in Knott et al. (1991, Statistician, 40, 217) can be represented as a tree model with one branch for response/non-response and another branch for correct/incorrect response, and each branch probability is characterized by an item response theory model. In the model, it is assumed that there is only one source of non-responses. However, in questionnaires or educational tests,

Extended redundancy analysis (ERA) is used to reduce multiple sets of predictors to a smaller number of components and examine the effects of these components on a response variable. In various social and behavioural studies, auxiliary covariates (e.g., gender, ethnicity) can often lead to heterogeneous subgroups of observations, each of which involves distinctive relationships between predictor and

In recent years a number of articles have focused on the identifiability of the basic local independence model. The identifiability issue usually concerns two model parameter sets predicting an identical probability distribution on the response patterns. Both parameter sets are applied to the same knowledge structure. However, nothing is known about cases where different knowledge structures predict

The three-parameter logistic model is widely used to model the responses to a proficiency test when the examinees can guess the correct response, as is the case for multiple-choice items. However, the weak identifiability of the parameters of the model results in large variability of the estimates and in convergence difficulties in the numerical maximization of the likelihood function. To overcome

We develop factor copula models to analyse the dependence among mixed continuous and discrete responses. Factor copula models are canonical vine copulas that involve both observed and latent variables, hence they allow tail, asymmetric and nonlinear dependence. They can be explained as conditional independence models with latent variables that do not necessarily have an additive latent structure. We

Principal covariate regression (PCOVR) is a method for regressing a set of criterion variables with respect to a set of predictor variables when the latter are many in number and/or collinear. This is done by extracting a limited number of components that simultaneously synthesize the predictor variables and predict the criterion ones. So far, no procedure has been offered for estimating statistical

Hick’s law, one of the few law-like relationships involving human performance, expresses choice reaction time as a linear function of the mutual information between the stimulus and response events. However, since this law was first proposed in 1952, its validity has been challenged by the fact that it only holds for the overall reaction time (RT) across all the stimuli, and does not hold for the reaction

Among current state-of-the-art estimation methods for multilevel IRT models, the two-stage divide-and-conquer strategy has practical advantages, such as clearer definition of factors, convenience for secondary data analysis, convenience for model calibration and fit evaluation, and avoidance of improper solutions. However, various studies have shown that, under the two-stage framework, ignoring measurement

Data in social sciences are typically non-normally distributed and characterized by heavy tails. However, most widely used methods in social sciences are still based on the analyses of sample means and sample covariances. While these conventional methods continue to be used to address new substantive issues, conclusions reached can be inaccurate or misleading. Although there is no ‘best method’ in

With advances in computerized tests, it has become commonplace to register not just the accuracy of the responses provided to the items, but also the response time. The idea that for each response both response accuracy and response time are indicative of ability has explicitly been incorporated in the signed residual time (SRT) model (Maris & van der Maas, 2012, Psychometrika, 77, 615–633), which

When scaling data using item response theory, valid statements based on the measurement model are only permissible if the model fits the data. Most item fit statistics used to assess the fit between observed item responses and the item responses predicted by the measurement model show significant weaknesses, such as the dependence of fit statistics on sample size and number of items. In order to assess

Let be the knowledge space derived from an attribution function σ on Q. Under an assumption for σ, this paper gives some necessary and sufficient conditions such that is discriminative. It also discusses the resolubility of σ when Q is an infinite set. More precisely, this paper proves that σ is not resoluble if Q is uncountable, and gives a necessary and sufficient condition such that σ is resoluble

In this paper it is shown that under the random effects generalized partial credit model for the measurement of a single latent variable by a set of polytomously scored items, the joint marginal probability distribution of the item scores has a closed‐form expression in terms of item category location parameters, parameters that characterize the distribution of the latent variable in the subpopulation

A three-level piecewise growth model (3L-PGM) can be used to break up nonlinear growth into multiple components, providing the opportunity to examine potential sources of variation in individual and contextual growth within different segments of the model. The conventional 3L-PGM assumes that the data are strictly hierarchical in nature, where measurement occasions (level 1) are nested within individuals

The Q-matrix identifies the subset of attributes measured by each item in the cognitive diagnosis modelling framework. Usually constructed by domain experts, the Q-matrix might contain some misspecifications, disrupting classification accuracy. Empirical Q-matrix validation methods such as the general discrimination index (GDI) and Wald have shown promising results in addressing this problem. However

Many theories have been put forward on how people become synchronized or co-regulate each other in daily interactions. These theories are often tested by observing a dyad and coding the presence of multiple target behaviours in small time intervals. The sequencing and co-occurrence of the partners’ behaviours across time are then quantified by means of association measures (e.g., kappa coefficient

In this article we provide an overview of existing approaches for relating latent class membership to external variables of interest. We extend on the work of Nylund‐Gibson et al. (Structural Equation Modeling: A Multidisciplinary Journal, 2019, 26, 967), who summarize models with distal outcomes by providing an overview of most recommended modeling options for models with covariates and larger models

In Van Rijn, P. W., & Ali, U. S. (2017), the second affiliation for Usama S. Ali was omitted and it should read as given below: South Valley University, Qena, Egypt. The author’s second affiliation has been added to the online article.