Methods for the treatment of item non‐response in attitudinal scales and in large‐scale assessments under the pairwise likelihood (PL) estimation framework and under a missing at random (MAR) mechanism are proposed. Under a full information likelihood estimation framework and MAR, ignorability of the missing data mechanism does not lead to biased estimates. However, this is not the case for pseudo‐likelihood

The non‐response model in Knott et al. (1991, Statistician, 40, 217) can be represented as a tree model with one branch for response/non‐response and another branch for correct/incorrect response, and each branch probability is characterized by an item response theory model. In the model, it is assumed that there is only one source of non‐responses. However, in questionnaires or educational tests,

Extended redundancy analysis (ERA) is used to reduce multiple sets of predictors to a smaller number of components and examine the effects of these components on a response variable. In various social and behavioural studies, auxiliary covariates (e.g., gender, ethnicity) can often lead to heterogeneous subgroups of observations, each of which involves distinctive relationships between predictor and

In recent years a number of articles have focused on the identifiability of the basic local independence model. The identifiability issue usually concerns two model parameter sets predicting an identical probability distribution on the response patterns. Both parameter sets are applied to the same knowledge structure. However, nothing is known about cases where different knowledge structures predict

The three‐parameter logistic model is widely used to model the responses to a proficiency test when the examinees can guess the correct response, as is the case for multiple‐choice items. However, the weak identifiability of the parameters of the model results in large variability of the estimates and in convergence difficulties in the numerical maximization of the likelihood function. To overcome

We develop factor copula models to analyse the dependence among mixed continuous and discrete responses. Factor copula models are canonical vine copulas that involve both observed and latent variables, hence they allow tail, asymmetric and nonlinear dependence. They can be explained as conditional independence models with latent variables that do not necessarily have an additive latent structure. We

Principal covariate regression (PCOVR) is a method for regressing a set of criterion variables with respect to a set of predictor variables when the latter are many in number and/or collinear. This is done by extracting a limited number of components that simultaneously synthesize the predictor variables and predict the criterion ones. So far, no procedure has been offered for estimating statistical

Hick’s law, one of the few law‐like relationships involving human performance, expresses choice reaction time as a linear function of the mutual information between the stimulus and response events. However, since this law was first proposed in 1952, its validity has been challenged by the fact that it only holds for the overall reaction time (RT) across all the stimuli, and does not hold for the reaction

Among current state‐of‐the‐art estimation methods for multilevel IRT models, the two‐stage divide‐and‐conquer strategy has practical advantages, such as clearer definition of factors, convenience for secondary data analysis, convenience for model calibration and fit evaluation, and avoidance of improper solutions. However, various studies have shown that, under the two‐stage framework, ignoring measurement

Data in social sciences are typically non‐normally distributed and characterized by heavy tails. However, most widely used methods in social sciences are still based on the analyses of sample means and sample covariances. While these conventional methods continue to be used to address new substantive issues, conclusions reached can be inaccurate or misleading. Although there is no ‘best method’ in

With advances in computerized tests, it has become commonplace to register not just the accuracy of the responses provided to the items, but also the response time. The idea that for each response both response accuracy and response time are indicative of ability has explicitly been incorporated in the signed residual time (SRT) model (Maris & van der Maas, 2012, Psychometrika, 77, 615–633), which

When scaling data using item response theory, valid statements based on the measurement model are only permissible if the model fits the data. Most item fit statistics used to assess the fit between observed item responses and the item responses predicted by the measurement model show significant weaknesses, such as the dependence of fit statistics on sample size and number of items. In order to assess

Let be the knowledge space derived from an attribution function σ on Q. Under an assumption for σ, this paper gives some necessary and sufficient conditions such that is discriminative. It also discusses the resolubility of σ when Q is an infinite set. More precisely, this paper proves that σ is not resoluble if Q is uncountable, and gives a necessary and sufficient condition such that σ is resoluble

In this paper it is shown that under the random effects generalized partial credit model for the measurement of a single latent variable by a set of polytomously scored items, the joint marginal probability distribution of the item scores has a closed‐form expression in terms of item category location parameters, parameters that characterize the distribution of the latent variable in the subpopulation

A three‐level piecewise growth model (3L‐PGM) can be used to break up nonlinear growth into multiple components, providing the opportunity to examine potential sources of variation in individual and contextual growth within different segments of the model. The conventional 3L‐PGM assumes that the data are strictly hierarchical in nature, where measurement occasions (level 1) are nested within individuals

The Q‐matrix identifies the subset of attributes measured by each item in the cognitive diagnosis modelling framework. Usually constructed by domain experts, the Q‐matrix might contain some misspecifications, disrupting classification accuracy. Empirical Q‐matrix validation methods such as the general discrimination index (GDI) and Wald have shown promising results in addressing this problem. However

Many theories have been put forward on how people become synchronized or co‐regulate each other in daily interactions. These theories are often tested by observing a dyad and coding the presence of multiple target behaviours in small time intervals. The sequencing and co‐occurrence of the partners’ behaviours across time are then quantified by means of association measures (e.g., kappa coefficient

In this article we provide an overview of existing approaches for relating latent class membership to external variables of interest. We extend on the work of Nylund‐Gibson et al. (Structural Equation Modeling: A Multidisciplinary Journal, 2019, 26, 967), who summarize models with distal outcomes by providing an overview of most recommended modeling options for models with covariates and larger models

In Van Rijn, P. W., & Ali, U. S. (2017), the second affiliation for Usama S. Ali was omitted and it should read as given below: South Valley University, Qena, Egypt. The author’s second affiliation has been added to the online article.

Multidimensional item response theory (MIRT) is widely used in assessment and evaluation of educational and psychological tests. It models the individual response patterns by specifying a functional relationship between individuals' multiple latent traits and their responses to test items. One major challenge in parameter estimation in MIRT is that the likelihood involves intractable multidimensional

Despite the long‐standing discussion on fixed effects (FE) and random effects (RE) models, how and under what conditions both methods can eliminate unmeasured confounding bias has not yet been widely understood in practice. Using a simple pretest–posttest design in a linear setting, this paper translates the conventional algebraic formalization of FE and RE models into causal graphs and provides intuitively

Previous designs for online calibration have only considered examinees’ responses to items. However, the use of response time, a useful metric that can easily be collected by a computer, has not yet been embedded in calibration designs. In this article we utilize response time to optimize the assignment of new items online, and accordingly propose two new adaptive designs. These are the D‐optimal per

The misclassification error distance and the adjusted Rand index are two of the most common criteria used to evaluate the performance of clustering algorithms. This paper provides an in‐depth comparison of the two criteria, with the aim of better understand exactly what they measure, their properties and their differences. Starting from their population origins, the investigation includes many data

Growth curve models have been widely used to analyse longitudinal data in social and behavioural sciences. Although growth curve models with normality assumptions are relatively easy to estimate, practical data are rarely normal. Failing to account for non‐normal data may lead to unreliable model estimation and misleading statistical inference. In this work, we propose a robust approach for growth

Rasch analysis is a popular statistical tool for developing and validating instruments that aim to measure human performance, attitudes and perceptions. Despite the availability of various software packages, constructing a good instrument based on Rasch analysis is still considered to be a complex, labour‐intensive task, requiring human expertise and rather subjective judgements along the way. In this

Using Louis’ formula, it is possible to obtain the observed information matrix and the corresponding large‐sample standard error estimates after the expectation–maximization (EM) algorithm has converged. However, Louis’ formula is commonly de‐emphasized due to its relatively complex integration representation, particularly when studying latent variable models. This paper provides a holistic overview

Estimates of subgroup differences are routinely used as part of a comprehensive validation system, and these estimates serve a critical role, including evaluating adverse impact. Unfortunately, under direct range restriction, a selected mean ( ) is a biased estimator of the population mean as well as the selected true score mean . This is due partly to measurement bias. This bias, as we show, is a

In exploratory factor analysis, latent factors and factor loadings are seldom interpretable until analytic rotation is performed. Typically, the rotation problem is solved by numerically searching for an element in the manifold of orthogonal or oblique rotation matrices such that the rotated factor loadings minimize a pre‐specified complexity function. The widely used gradient projection (GP) algorithm

This study proposes and evaluates a diagnostic classification model framework for multiple‐choice items. Models in the proposed framework have a two‐level nested structure which allows for binary scoring (for correctness) and polytomous scoring (for distractors) at the same time. One advantage of these models is that they can provide distractor information while maintaining the statistical properties

A test score on a psychological test is usually expressed as a normed score, representing its position relative to test scores in a reference population. These typically depend on predictor(s) such as age. The test score distribution conditional on predictors is estimated using regression, which may need large normative samples to estimate the relationships between the predictor(s) and the distribution

Computer simulations have become a popular tool for assessing complex skills such as problem‐solving. Log files of computer‐based items record the human–computer interactive processes for each respondent in full. The response processes are very diverse, noisy, and of non‐standard formats. Few generic methods have been developed to exploit the information contained in process data. In this paper we

In this paper we derive optimal designs for the Rasch Poisson counts model and its extended version of the (generalized) negative binomial counts model incorporating several binary predictors for the difficulty parameter. To efficiently estimate the regression coefficients of the predictors, locally D‐optimal designs are developed. After an introduction to the Rasch Poisson counts model and its extension

Recently, a multiple comparisons procedure was derived with the goal of determining whether it is reasonable to make a decision about which of J independent groups has the largest robust measure of location. This was done by testing hypotheses aimed at comparing the group with the largest estimate to the remaining J − 1 groups. It was demonstrated that for the goal of controlling the familywise error

Computerized classification testing (CCT) aims to classify persons into one of two or more possible categories to make decisions such as mastery/non‐mastery or meet most/meet all/exceed. A defining feature of CCT is its stopping criterion: the test terminates when there is enough confidence to make a decision. There is abundant research on CCT with a single cut‐off, and two common stopping criteria

The design of recommendation strategies in the adaptive learning systems focuses on utilizing currently available information to provide learners with individual‐specific learning instructions. As a critical motivate for human behaviours, curiosity is essentially the drive to explore knowledge and seek information. In a psychologically inspired view, we propose a curiosity‐driven recommendation policy

Determining a lack of association between an outcome variable and a number of different explanatory variables is frequently necessary in order to disregard a proposed model (i.e., to confirm the lack of a meaningful association between an outcome and predictors). Despite this, the literature rarely offers information about, or technical recommendations concerning, the appropriate statistical methodology

Quantiles are widely used in both theoretical and applied statistics, and it is important to be able to deploy appropriate quantile estimators. To improve performance in the lower and upper quantiles, especially with small sample sizes, a new quantile estimator is introduced which is a weighted average of all order statistics. The new estimator, denoted NO, has desirable asymptotic properties. Moreover

Ordinal predictors are commonly used in regression models. They are often incorrectly treated as either nominal or metric, thus under‐ or overestimating the information contained. Such practices may lead to worse inference and predictions compared to methods which are specifically designed for this purpose. We propose a new method for modelling ordinal predictors that applies in situations in which

Psychological theories often produce hypotheses that pertain to individual differences in within‐person variability. To empirically test the predictions entailed by such hypotheses with longitudinal data, researchers often use multilevel approaches that allow them to model between‐person differences in the mean level of a certain variable and the residual within‐person variance. Currently, these approaches

We propose a latent topic model with a Markov transition for process data, which consists of time‐stamped events recorded in a log file. Such data are becoming more widely available in computer‐based educational assessment with complex problem‐solving items. The proposed model can be viewed as an extension of the hierarchical Bayesian topic model with a hidden Markov structure to accommodate the underlying

In the multilevel modelling literature, methodologists widely acknowledge that a level‐1 variable can have distinct within‐cluster and between‐cluster effects, and that failing to disaggregate these can yield a slope estimate that is an uninterpretable, conflated blend of the two. Methodologists have stated, however, that including conflated slopes of level‐1 variables in a model is not problematic

In a cognitive diagnostic assessment (CDA), attributes refer to fine-grained knowledge points or skills. The Q-matrix is a central component of CDA, which specifies the relationship between items and attributes. Oftentimes, attributes and Q-matrix are defined by subject-matter experts, and assumed to be appropriate without any misspecifications. However, this assumption does not always hold in real

In low-stakes assessments, test performance has few or no consequences for examinees themselves, so that examinees may not be fully engaged when answering the items. Instead of engaging in solution behaviour, disengaged examinees might randomly guess or generate no response at all. When ignored, examinee disengagement poses a severe threat to the validity of results obtained from low-stakes assessments

Deterministic blockmodelling is a well‐established clustering method for both exploratory and confirmatory social network analysis seeking partitions of a set of actors so that actors within each cluster are similar with respect to their patterns of ties to other actors (or, in some cases, other objects when considering two‐mode networks). Even though some of the historical foundations for certain

Longitudinal studies are the gold standard for research on time-dependent phenomena in the social sciences. However, they often entail high costs due to multiple measurement occasions and a long overall study duration. It is therefore useful to optimize these design factors while maintaining a high informativeness of the design. Von Oertzen and Brandmaier (2013, Psychology and Aging, 28, 414) applied

The point-biserial correlation is a commonly used measure of effect size in two-group designs. New estimators of point-biserial correlation are derived from different forms of a standardized mean difference. Point-biserial correlations are defined for designs with either fixed or random group sample sizes and can accommodate unequal variances. Confidence intervals and standard errors for the point-biserial

The four-parameter logistic model (4PLM) has recently attracted much interest in various applications. Motivated by recent studies that re-express the four-parameter model as a mixture model with two levels of latent variables, this paper develops a new expectation-maximization (EM) algorithm for marginalized maximum a posteriori estimation of the 4PLM parameters. The mixture modelling framework of

Most partitioning methods used in psychological research seek to produce homogeneous groups (i.e., groups with low intra‐group dissimilarity). However, there are also applications where the goal is to provide heterogeneous groups (i.e., groups with high intra‐group dissimilarity). Examples of these anticlustering contexts include construction of stimulus sets, formation of student groups, assignment

The use of hierarchical data (also called multilevel data or clustered data) is common in behavioural and psychological research when data of lower-level units (e.g., students, clients, repeated measures) are nested within clusters or higher-level units (e.g., classes, hospitals, individuals). Over the past 25 years we have seen great advances in methods for computing the sample sizes needed to obtain

According to Wollack and Schoenig (2018, The Sage encyclopedia of educational research, measurement, and evaluation. Thousand Oaks, CA: Sage, 260), benefiting from item preknowledge is one of the three broad types of test fraud that occur in educational assessments. We use tools from constrained statistical inference to suggest a new statistic that is based on item scores and response times and can

Count data naturally arise in several areas of cognitive ability testing, such as processing speed, memory, verbal fluency, and divergent thinking. Contemporary count data item response theory models, however, are not flexible enough, especially to account for over- and underdispersion at the same time. For example, the Rasch Poisson counts model (RPCM) assumes equidispersion (conditional mean and

Intensive longitudinal studies are becoming progressively more prevalent across many social science areas, and especially in psychology. New technologies such as smart‐phones, fitness trackers, and the Internet of Things make it much easier than in the past to collect data for intensive longitudinal studies, providing an opportunity to look deep into the underlying characteristics of individuals under

Personality constructs, attitudes and other non‐cognitive variables are often measured using rating or Likert‐type scales, which does not come without problems. Especially in low‐stakes assessments, respondents may produce biased responses due to response styles (RS) that reduce the validity and comparability of the measurement. Detecting and correcting RS is not always straightforward because not

Many educational and psychological assessments focus on multidimensional latent traits that often have a hierarchical structure to provide both overall‐level information and fine‐grained diagnostic information. A test will usually have either separate time limits for each subtest or an overall time limit for administrative convenience and test fairness. In order to complete the items within the allocated

Given a drift diffusion model with unknown drift and boundary parameters, we analyse the behaviour of maximum likelihood estimates with respect to changes of responses and response times. It is shown analytically that a single fast response time can dominate the estimation in that no matter how many correct answers a test taker provides, the estimate of the drift (ability) parameter decreases to zero

In recent years, item response tree (IRTree) approaches have received increasing attention in the response style literature for their ability to partial out response style latent variables as well as associated item parameters. When an IRTree approach is adopted to measure extreme response styles, directional and content invariance could be assumed at the latent variable and item parameter levels.