Although measures such as sensitivity and specificity are used in the study of diagnostic test accuracy, these are not appropriate for integrating heterogeneous studies. Therefore, it is essential to assess in detail all related aspects prior to integrating a set of studies so that the correct model can then be selected. This work describes the scheme employed for making decisions regarding the use

Methods to assess measurement invariance in constructs have received much attention, as invariance is critical for accurate group comparisons. Less attention has been given to the identification and correction of the sources of non-invariance in predictive equations. This work developed correction factors for structural intercept and slope bias in common regression equations to address calls in the

This article addresses the comparability of sampling and fieldwork with an analysis of methodological data describing 1,537 national surveys from five major comparative cross-national survey projects in Europe carried out in the period from 1981 to 2017. We describe the variation in the quality of the survey documentation, and in the survey methodologies themselves, focusing on survey procedures with

This paper examined the amount bias in standard errors for fixed effects when the random part of a multilevel model is misspecified. Study 1 examined the effects of misspecification for a model with one Level 1 predictor. Results indicated that misspecifying random slope variance as fixed had a moderate effect size on the standard errors of the fixed effects and had a greater effect than misspecifying

The present research examined the distributional properties of construct reliability indices and model fit metrics, explored relationships between and among the indices and metrics, and investigated variables influencing the relative magnitudes of the indices and metrics in structural equation measurement models. A broad-based meta-analysis of reported construct reliability indices and selected model

Data mining is one of the most effective statistical methodologies to investigate a variety of problems in areas including pattern recognition, machine learning, bioinformatics, chemometrics, and statistics. In particular, statistically-sophisticated procedures that emphasize on reliability of results and computational efficiency are required for the analysis of high-dimensional data. Optimization

Linear Mixed Effect Models (LMEM) have become a popular method for analyzing nested experimental data, which are often encountered in psycholinguistics and other fields. This approach allows experimental results to be generalized to the greater population of both subjects and experimental stimuli. In an influential paper Bar and his colleagues (2013; https://doi.org/10.1016/j.jml.2012.11.001) recommend

Previous research applying multilevel models to single-case data has made a critical assumption that the level-1 error covariance matrix is constant across all participants. However, the level-1 error covariance matrix may differ across participants and ignoring these differences can have an impact on estimation and inferences. Despite the importance of this issue, the effects of modeling between-case

Approaches for dealing with item omission include incorrect scoring, ignoring missing values, and approaches for nonignorable missing values and have only been evaluated for certain forms of nonignorability. In this paper we investigate the performance of these approaches for various conditions of nonignorability, that is, when the missing response depends on i) the item response, ii) a latent missing

Confidence intervals (CIs) constitute the most popular alternative to widely criticized null hypothesis significance tests. CIs provide more information than significance tests and lend themselves well to visual displays. Although CIs are no better than significance tests when used solely as significance tests, researchers need not limit themselves to this use of CIs. Rather, CIs can be used to estimate

Psychological research is confronted with ever-increasing demands to save resources such as time and money while assuring high ethical standards. In medical and pharmaceutical research, group sequential designs have fundamentally changed traditional statistical testing approaches featuring only one analysis at the end of a single-stage study. They enable early stopping at an interim stage, after a

Nonlinear structural equation models within the frequentist framework were developed to work with continuous items. Applied researchers who usually work with Likert-type items choose between two strategies to estimate such models: treat items as continuous variables or create item parcels. Two Monte Carlo studies were conducted to evaluate the effects of each strategy on estimates and Type I errors

Since factor analysis is one of the most often used techniques in psychometrics, comparing or combining solutions from different factor analyses is often needed. Several measures to compare factors exist, one of the best known is Tucker’s congruence coefficient, which is enjoying newly found popularity thanks to the recent work of Lorenzo-Seva and ten Berge (2006), who established cut-off values for

Open survey questions are often used to evaluate closed questions. However, they can fulfil this function only if there is a strong link between answers to open questions and answers to related closed questions. Using reasons for non-voting reported in the German Longitudinal Election Study 2013, we investigated this link by examining whether the reported reasons for non-voting may be substantive reasons

Institutional trust is in decline in many western democracies. Since the 2008 global economic and financial crisis, this increasing distrust has been closely related to trust in political institutions. Trust in institutions is one of the pillars of democracy, and its decline is one of the most evident and shared symptoms of the recession, especially in those contexts where it has been particularly

Abstract. The paper describes EV scaling for variances of latent variables included in confirmatory factor models. EV-scaled variances can be achieved in two ways: the estimation of variance parame...

Abstract. Drawing valid inferences about daily or long-term dynamics of psychological constructs (e.g., depression) requires the measurement model (indicating which constructs are measured by which...

Abstract. Building on the stochastic theory of causal effects and latent state-trait theory, this article shows how a comprehensive analysis of the effects of interventions can be conducted based o...

Abstract. The social relations model (SRM) is typically used to identify sources of variance in interpersonal dispositions in families. Traditionally, it uses dyadic measurements that are obtained ...

Abstract. Maximum Likelihood (ML) estimation is a common estimation method in Structural Equation Modeling (SEM), and parameters in such analyses are interpreted using frequentist terms and definit...

Abstract. Single-case experimental designs (SCEDs) are interrupted time-series designs that have recently gained recognition as being able to provide a strong basis for establishing intervention ef...

Abstract. In psychological tests, the time needed to respond to the items provides collateral information about the latent traits of the test takers. This, however, requires a measurement model tha...

Abstract. The concept of replication is fundamental to the logic and rhetoric of science, including the argument that science is self-correcting. Yet there is very little literature on the methodol...

Abstract. It is challenging to apply exploratory factor analysis (EFA) to event-related potential (ERP) data because such data are characterized by substantial temporal overlap (i.e., large cross-l...

Abstract. Equivalence tests (also known as similarity or parity tests) have become more and more popular in addition to equality tests. However, in testing the equivalence of two population means, ...

Abstract. This article proposes an approach, intended for factor-analytic (FA) psychometric applications, which aims to assess the extent to which the FA-derived individual score estimates are accu...

Abstract. When planning group-randomized studies probing mediation, effective and efficient sample allocation is governed by several parameters including treatment-mediator and mediator-outcome pat...

Abstract. In a recent paper, Ferrer and Pardo (2014) tested several distribution-based methods designed to assess when test scores obtained before and after an intervention reflect a statistically ...

Abstract. Cognitive diagnostic modeling has been adopted to support various diagnostic measuring processes. Specifically, this approach allows practitioners and/or researchers to investigate an ind...

Abstract. Multilevel models have recently been used to empirically investigate the idea that social characteristics are intersectional such as age, sex, ethnicity, and socioeconomic position intera...

Abstract. Bi-factor exploratory modeling has recently emerged as a promising approach to multidimensional psychological measurement. However, state-of-the-art methods relying on target rotation req...

Abstract. Research, policy, and practice often target specific subpopulations. Large-scale survey studies are particularly useful for performing subpopulation analyses due to the large and represen...

Abstract. We aimed to evaluate the impact of two question wordings, one more directive (reference, European Social Survey) and one less directive (modified version), on self-reported trust. Using a...

Longitudinal developmental research is often focused on patterns of change or growth across different (sub)groups of individuals. Particular to some research contexts, developmental inquiries may involve one or more (sub)groups that are small in nature and therefore difficult to properly capture through statistical analysis. The current study explores the lower-bound limits of subsample sizes in a

Factorial survey experiments are increasingly used in the social sciences to investigate behavioral intentions. The measurement of self-reported behavioral intentions with factorial survey experiments frequently assumes that the determinants of intended behavior affect actual behavior in a similar way. We critically investigate this fundamental assumption using the misdirected email technique. Student

Abstract. Response bias (nonresponse and social desirability bias) is one of the main concerns when asking sensitive questions about behavior and attitudes. Self-reports on sensitive issues as in h...

Factorial validity is often assessed using confirmatory factor analysis. Model fit is commonly evaluated using the cutoff values for the fit indices proposed by Hu and Bentler (1999). There is a body of research showing that those cutoff values cannot be generalized. Model fit does not only depend on the severity of misspecification, but also on nuisance parameters, which are independent of the misspecification

The investigation of differential item functioning (DIF) is important for any group comparison because the validity of the inferences made from scale scores could be compromised if DIF is present. DIF occurs when individuals from different groups show different probabilities of selecting a response option to an item after being matched on the underlying latent variable that the item is supposed to

Abstract. In the construction of scales intended for the use in cross-cultural studies, the selection of items needs to be guided not only by traditional criteria of item quality, but has to take i...

Bollen and colleagues have advocated the use of formative scales despite the fact that formative scales lack an adequate underlying theory to guide development or validation such as that which underlies reflective scales. Three conceptual impediments impede the development of such theory: the redefinition of measurement restricted to the context of model fitting, the inscrutable notion of conceptual

The purpose of this work is to identify issues regarding the concept of validity in educational and psychological testing on which there is, and is not, consensus among experts, using an online Delphi study. Although many theorists have expressed their views about the proper characterization of validity, it is important to systematically collect ideas about each aspect of validity. Study participants

Abstract. In this article, a new model is proposed for the responses and the response times in attitudinal or personality inventories with graded response format. The model is based on the lognorma...

Abstract. Extending from classical test theory, G theory allows more sources of variations to be investigated and therefore provides the accuracy of generalizing observed scores to a broader univer...

The mixed-effects location scale model is an extension of a multilevel model for longitudinal data. It allows covariates to affect both the within-subject variance and the between-subject variance (i.e., the intercept variance) beyond their influence on the means. Typically, the model is applied to two-level data (e.g., the repeated measurements of persons), although researchers are often faced with

The use of agree/disagree (A/D) questions is a common technique to measure attitudes. For instance, this question format is employed frequently in the Eurobarometer and International Social Survey Programme (ISSP). Theoretical considerations, however, suggest that A/D questions require a complex processing. Therefore, many survey researchers have recommended the use of item-specific (IS) questions

When using the mixture Rasch model, the model identification constraints are either to set the equal means for all classes in the assumed normal ability distributions (equal ability mean constraint in short), or to set the sum of item difficulties to be zero for each class. In real data analysis, however, both constraints are not always sufficient to establish a common scale across latent classes unless

Circular data is different from linear data and its analysis also requires methods different from conventional methods. In this study a Bayesian embedding approach to estimating circular regression models is investigated, by means of simulation studies, in terms of performance, efficiency, and flexibility. A new Markov chain Monte Carlo (MCMC) sampling method is proposed and contrasted to an existing

Latent class analysis has been recently proposed for the multiple imputation (MI) of missing categorical data, using either a standard frequentist approach or a nonparametric Bayesian model called Dirichlet process mixture of multinomial distributions (DPMM). The main advantage of using a latent class model for multiple imputation is that it is very flexible in the sense that it can capture complex

In a discrete-time survival model the occurrence of some event is measured by the end of each time interval. In practice it is not always possible to measure all subjects at the same point in time. In this study the consequences of varying measurement occasions are investigated by means of a simulation study and the analysis of data from an empirical study. The results of the simulation study suggest

Predictive mean matching (PMM) is a state-of-the-art hot deck multiple imputation (MI) procedure. The quality of its results depends, inter alia, on the availability of suitable donor cases. Applying PMM in small sample scenarios often found in psychological or medical research could be problematic, as there might not be many (or any) suitable donor cases in the data set. So far, there has not been

The current study proposed a new model, termed the cross-classified multiple membership latent variable regression (CCMM-LVR) model that provides an extension to the three-level latent variable regression (HM3-LVR) model that can be used with cross-classified multiple membership data, for example, in the presence of student mobility across schools. The HM3-LVR model is beneficial for testing more flexible

Because survey response rates are consistently declining worldwide, survey researchers strive to obtain as much auxiliary information on sampled units as possible. Surveys using in-person interviewing often request that interviewers collect observations on key features of all sampled units, given that interviewers are the eyes and ears of the survey organization. Unfortunately, these observations are

The present article features a hierarchical Bayes method applied to solving problems of benchmarking and contemporaneous reconciliation across time series. This method enables the use of high frequency series to be either approximations or one or several related indicators. This method may be applied when facing flow or index disaggregation problems. The authors compare their results to classical procedures

The bifactor model (BM) and the testlet response model (TRM) are the most common multidimensional models applied to testlet-based tests. The common procedure is to estimate these models using different estimation methods (see, e.g., DeMars, 2006). A possible consequence of this is that previous findings about the implications of fitting a wrong model to the data may be confounded with the estimation

The present study considered two independent exponential distributions with the hypothesis for testing equivalence of lifetime means or failure rates, and aimed to determine the required uncensored sample size based on power, sampling cost, and censoring proportion simultaneously in the case of right censoring. Approximate sample size formulas with an iterative procedure were proposed and an uncensored

The form-resistant hypothesis states that alternative ways of measuring the same values should be small if method-specific features are taken into account. However, previous research that compared rating and ranking questionnaires for measuring values has shown mixed results. We suggest that adopting a latent class segmentation approach helps to explain these mixed results by identifying segments with

In repeated cross-sections of organizations, different individuals are sampled from the same set of organizations at each time point of measurement. As a result, common longitudinal data analysis methods (e.g., latent growth curve models) cannot be applied in the usual way. In this contribution, a multilevel structural equation modeling approach to analyze data from repeated cross-sections is presented

Growth curve modeling (GCM) is an important and commonly used methodology in the social sciences for examining change over time in a variable value. While much of the empirical research examining the performance of various estimators under a variety of conditions has focused on continuous (and typically normally distributed) observed indicators, in practice researchers frequently make use of categorical

Abstract. There has been an increase of interest in psychometric models referred to as cognitive diagnosis models (CDMs). A critical concern is in selecting the most appropriate model at the item level. Several tests for model comparison have been employed, which include the likelihood ratio (LR) and the Wald (W) tests. Although the LR test is relatively more robust than the W test, the current implementation

Abstract. This article reviews computational social science methods and their relation to conventional methodology and statistics. Computational social science has three important features. Firstly, it often involves big data; data sets so large that conventional database and analysis techniques cannot handle them with ease. Secondly, dealing with these big data sets has given rise to analysis techniques