Understanding the longitudinal dynamics of behavior, their stability and change over time, are of great interest in the social and behavioral sciences. Researchers investigate the degree to which an observed measure reflects stable components of the construct, situational fluctuations, method effects, or just random measurement error. An important question in such models is whether autoregressive effects

In recent years, psychological research has faced a credibility crisis, and open data are often regarded as an important step toward a more reproducible psychological science. However, privacy concerns are among the main reasons that prevent data sharing. Synthetic data procedures, which are based on the multiple imputation (MI) approach to missing data, can be used to replace sensitive data with simulated

Next Eigenvalue Sufficiency Test (NEST; Achim, 2017) is a recently proposed method to determine the number of factors in exploratory factor analysis (EFA). NEST sequentially tests the null-hypothesis that k factors are sufficient to model correlations among observed variables. Another recent approach to detect factors is exploratory graph analysis (EGA; Golino & Epskamp, 2017), which rules the number

The present tutorial proposes to use concepts of physics and mathematics to help behavioral scientists to use differential equations in their studies. It focuses on the first-order and the second-order (damped oscillator) differential equation. Simple examples allow to detail the meaning of the coefficients, the conditions of applicability of these differential equations, the underlying hypothesis

In 2019 we wrote an article (Tendeiro & Kiers, 2019) in Psychological Methods over null hypothesis Bayesian testing and its working horse, the Bayes factor. Recently, van Ravenzwaaij and Wagenmakers (2021) offered a response to our piece, also in this journal. Although we do welcome their contribution with thought-provoking remarks on our article, we ended up concluding that there were too many “issues”

I compared multiple methods to estimate and test a difference in correlations (ρdiff) between two variables that are repeatedly measured or originate from dyads. Fisher’s z transformed correlations are often used for testing ρdiff. However, raw scores are typically used directly to compute correlations under this popular method, whose performance has not been evaluated with measurement error or nonnormality

In modern test theory, response variables are a function of a common latent variable that represents the measured attribute, and error variables that are unique to the response variables. While considerable thought goes into the interpretation of latent variables in these models (e.g., validity research), the interpretation of error variables is typically left implicit (e.g., describing error variables

One primary advantage of receiver operating characteristic (ROC) analysis is considered to be its ability to quantify classification performance independently of factors such as prior probabilities and utilities of classification outcomes. This article argues the opposite. When evaluating classification performance, ROC analysis should consider prior probabilities and utilities. By developing expected

Replication is central to scientific progress. Because of widely reported replication failures, replication has received increased attention in psychology, sociology, education, management, and related fields in recent years. Replication studies have generally been assessed dichotomously, designated either a “success” or “failure” based entirely on the outcome of a null hypothesis significance test

Almost always, developmental processes are multivariate in nature such that several outcomes and the development among these variables are correlated; therefore, empirical researchers often desire to examine two or more variables over time to understand how these outcomes and their change patterns are correlated. Multivariate growth models (MGMs) allow researchers to examine the correlations among

Extreme groups design (EGD) refers to the use of a screening variable to inform further data collection, such that only participants with the lowest and highest scores are recruited in subsequent stages of the study. It is an effective way to improve the power of a study under a limited budget, but produces biased standardized estimates. We demonstrate that the bias in EGD results from its inherent

Though much effort is often put into designing psychological studies, the measurement model and scoring approach employed are often an afterthought, especially when short survey scales are used (Flake & Fried, 2020). One possible reason that measurement gets downplayed is that there is generally little understanding of how calibration/scoring approaches could impact common estimands of interest, including

Latent transition analysis (LTA), also referred to as latent Markov modeling, is an extension of latent class/profile analysis (LCA/LPA) used to model the interrelations of multiple latent class variables. LTA methods have become increasingly accessible and in-turn are being utilized in applied research. The current article provides an introduction to LTA by answering 10 questions commonly asked by

Since the start of the 21st century, few advances have had as far-reaching impact in science as the widespread adoption of artificial neural networks in fields as diverse as fundamental physics, clinical medicine, and psychology. In research methods, one promising area for the adoption of artificial neural networks involves the analysis of single-case experimental designs. Given that these types of

Spectral clustering is a well-known method for clustering the vertices of an undirected network. Although its use in network psychometrics has been limited, spectral clustering has a close relationship to the commonly used walktrap algorithm. In this article, we report results from simulation experiments designed to evaluate the ability of spectral clustering and the walktrap algorithm to recover underlying

We present a general framework for causal mediation analysis using nonparametric Bayesian methods in the potential outcomes framework. Our model, which we refer to as the Bayesian causal mediation forests model, combines recent advances in Bayesian machine learning using decision tree ensembles, Bayesian nonparametric causal inference, and a Bayesian implementation of the g-formula for computing causal

Assessing measurement invariance is an important step in establishing a meaningful comparison of measurements of a latent construct across individuals or groups. Most recently, moderated nonlinear factor analysis (MNLFA) has been proposed as a method to assess measurement invariance. In MNLFA models, measurement invariance is examined in a single-group confirmatory factor analysis model by means of

Researchers use consensus emergence models (CEMs) to detect when the scores of group members become similar over time. The purpose of this article is to review how CEMs often lead to spurious conclusions of consensus emergence due to the problem of distinguishability, or the notion that different data-generating mechanisms sometimes give rise to similar observed data. As a result, CEMs often cannot

A regression model of predictor trade-offs is described. Each regression parameter equals the expected change in Y obtained by trading 1 point from one predictor to a second predictor. The model applies to predictor variables that sum to a constant T for all observations; for example, proportions summing to T = 1.0 or percentages summing to T = 100 for each observation. If predictor variables sum to

Single-case experimental designs (SCEDs) are used to study the effects of interventions on the behavior of individual cases, by making comparisons between repeated measurements of an outcome under different conditions. In research areas where SCEDs are prevalent, there is a need for methods to synthesize results across multiple studies. One approach to synthesis uses a multilevel meta-analysis (MLMA)

Most researchers have estimated the edge weights for relative importance networks using a well-established measure of general dominance for multiple regression. This approach has several desirable properties including edge weights that represent R² contributions, in-degree centralities that correspond to R² for each item when using other items as predictors, and strong replicability. We endorse the

Cross-lagged models are by far the most commonly used method to test the prospective effect of one construct on another, yet there are no guidelines for interpreting the size of cross-lagged effects. This research aims to establish empirical benchmarks for cross-lagged effects, focusing on the cross-lagged panel model (CLPM) and the random intercept cross-lagged panel model (RI-CLPM). We drew a quasirepresentative

This note contains a corrective and a generalization of results by Borsboom et al. (2008), based on Heesen and Romeijn (2019). It highlights the relevance of insights from psychometrics beyond the context of psychological testing.

Psychologists have become increasingly interested in the geographical organization of psychological phenomena. Such studies typically seek to identify geographical variation in psychological characteristics and examine the causes and consequences of that variation. Geo-psychological research offers unique advantages, such as a wide variety of easily obtainable behavioral outcomes. However, studies

Researchers are usually interested in examining the impact of covariates when separating heterogeneous samples into latent classes that are more homogeneous. The majority of theoretical and empirical studies with such aims have focused on identifying covariates as predictors of class membership in the structural equation modeling framework. In other words, the covariates only indirectly affect the

A scale to measure a psychological construct is subject to measurement error. When meta-analyzing correlations obtained from scale scores, many researchers recommend correcting for measurement error. I considered three caveats when correcting for measurement error in meta-analysis of correlations: (a) the distribution of true scores can be non-normal, resulting in violation of the normality assumption

When randomized control trials are not available, regression discontinuity (RD) designs are a viable quasi-experimental method shown to be capable of producing causal estimates of how a program or intervention affects an outcome. While the RD design and many related methodological innovations came from the field of psychology, RDs are underutilized among psychologists even though many interventions

Meta-analysis is an important quantitative tool for cumulative science, but its application is frustrated by publication bias. In order to test and adjust for publication bias, we extend model-averaged Bayesian meta-analysis with selection models. The resulting robust Bayesian meta-analysis (RoBMA) methodology does not require all-or-none decisions about the presence of publication bias, can quantify

Measurement invariance—the notion that the measurement properties of a scale are equal across groups, contexts, or time—is an important assumption underlying much of psychology research. The traditional approach for evaluating measurement invariance is to fit a series of nested measurement models using multiple-group confirmatory factor analyses. However, traditional approaches are strict, vary across

In the social sciences, measurement scales often consist of ordinal items and are commonly analyzed using factor analysis. Either data are treated as continuous, or a discretization framework is imposed in order to take the ordinal scale properly into account. Correlational analysis is central in both approaches, and we review recent theory on correlations obtained from ordinal data. To ensure appropriate

Regression models with interaction terms are common models for moderating relationships. When effects of several predictors from one group—for example, genetic variables—are potentially moderated by several predictors from another—for example, environmental variables—many interaction terms result. This complicates model interpretation, especially when coefficient signs point in different directions

When multiple hypothesis tests are conducted, the familywise Type I error probability correspondingly increases. Various multiple test procedures (MTPs) have been developed, which generally aim to control the familywise Type I error rate at the desired level. However, although multiplicity is frequently discussed in the ANOVA literature and MTPs are correspondingly employed, the issue has received

Growth mixture models (GMMs) are a popular method to identify latent classes of growth trajectories. One shortcoming of GMMs is nonconvergence, which often leads researchers to apply covariance equality constraints to simplify estimation, though this may be a dubious assumption. Alternative model specifications have been proposed to reduce nonconvergence without imposing covariance equality constraints

Single case experimental designs (SCEDs) are used to test treatment effects in a wide range of fields and consist of repeated measurements for a single case throughout one or more baseline phases and throughout one or more treatment phases. Recently, mediation analysis has been applied to SCEDs. Mediation analysis decomposes the total treatment-outcome effect into a direct and indirect effect, and

We introduce a new meta-analysis estimator, the weighted and iterated least squares (WILS), that greatly reduces publication selection bias (PSB) when selective reporting for statistical significance (SSS) is present. WILS is the simple weighted average that has smaller bias and rates of false positives than conventional meta-analysis estimators, the unrestricted weighted least squares (UWLS), and

Instead of using manifest proxies for a latent outcome or latent covariates in a causal effect analysis, the R package EffectLiteR facilitates a direct integration of latent variables based on structural equation models (SEM). The corresponding framework considers latent interactions and provides various effect estimates for evaluating the differential effectiveness of treatments. In addition, a user-friendly

Complexity science and systems thinking are increasingly recognized as relevant paradigms for studying systems where biology, psychology, and socioenvironmental factors interact. The application of systems thinking, however, often stops at developing a conceptual model that visualizes the mapping of causal links within a system, e.g., a causal loop diagram (CLD). While this is an important contribution

Publication bias poses a challenge for accurately synthesizing research findings using meta-analysis. A number of statistical methods have been developed to combat this problem by adjusting the meta-analytic estimates. Previous studies tended to apply these methods without regard to optimal conditions for each method’s performance. The present study sought to estimate the typical effect size attenuation

Various theoretical accounts suggest that within-person effects relating to everyday experiences (assessed, e.g., via experience sampling studies or daily diary studies) are a central element for understanding between-person differences in future outcomes. In this regard, it is often assumed that the within-person effect of a time-varying predictor X on a time-varying mediator M contributes to the

Model comparison is the cornerstone of theoretical progress in psychological research. Common practice overwhelmingly relies on tools that evaluate competing models by balancing in-sample descriptive adequacy against model flexibility, with modern approaches advocating the use of marginal likelihood for hierarchical cognitive models. Cross-validation is another popular approach but its implementation

Eye-tracking has emerged as a popular method for empirical studies of cognitive processes across multiple substantive research areas. Eye-tracking systems are capable of automatically generating fixation-location data over time at high temporal resolution. Often, the researcher obtains a binary measure of whether or not, at each point in time, the participant is fixating on a critical interest area

Various theoretical accounts suggest that within-person effects relating to everyday experiences (assessed, e.g., via experience sampling studies or daily diary studies) are a central element for understanding between-person differences in future outcomes. In this regard, it is often assumed that the within-person effect of a time-varying predictor X on a time-varying mediator M contributes to the

Network psychometrics is undergoing a time of methodological reflection. In part, this was spurred by the revelation that ℓ₁-regularization does not reduce spurious associations in partial correlation networks. In this work, we address another motivation for the widespread use of regularized estimation: the thought that it is needed to mitigate overfitting. We first clarify important aspects of overfitting

Bayesian t tests have become increasingly popular alternatives to null-hypothesis significance testing (NHST) in psychological research. In contrast to NHST, they allow for the quantification of evidence in favor of the null hypothesis and for optional stopping. A major drawback of Bayesian t tests, however, is that error probabilities of statistical decisions remain uncontrolled. Previous approaches

Bayesian hypothesis testing procedures have gained increased acceptance in recent years. A key advantage that Bayesian tests have over classical testing procedures is their potential to quantify information in support of true null hypotheses. Ironically, default implementations of Bayesian tests prevent the accumulation of strong evidence in favor of true null hypotheses because associated default

Researchers across varied fields increasingly are collecting and analyzing intensive longitudinal data (ILD) to examine processes across time at the individual level. Two types of relations are typically examined: lagged and contemporaneous. Lagged relations capture how variables at a prior time point can be used to explain variance in variables at a later time point. These are always modeled using

Multimethod measurement designs with planned missing data (MMM-PMD) aim at combining cheap proxy methods (e.g., self-reports) with an expensive gold standard method (e.g., biomarker) in order to improve the cost-efficiency of research designs. This article presents a comprehensive simulation study investigating whether accurate factor scores with trustworthy confidence or credible intervals for the

Statistical network models describing multivariate dependency structures in psychological data have gained increasing popularity. Such comparably novel statistical techniques require specific guidelines to make them accessible to the research community. So far, researchers have provided tutorials guiding the estimation of networks and their accuracy. However, there is currently little guidance in determining

Network approaches to psychometric constructs, in which constructs are modeled in terms of interactions between their constituent factors, have rapidly gained popularity in psychology. Applications of such network approaches to various psychological constructs have recently moved from a descriptive stance, in which the goal is to estimate the network structure that pertains to a construct, to a more

Network approaches to psychometric constructs, in which constructs are modeled in terms of interactions between their constituent factors, have rapidly gained popularity in psychology. Applications of such network approaches to various psychological constructs have recently moved from a descriptive stance, in which the goal is to estimate the network structure that pertains to a construct, to a more

Concerns about replication failures can be partially recast as concerns about excessive heterogeneity in research results. Although this heterogeneity is an inherent part of science (e.g., sampling variability; studying different conditions), not all heterogeneity results from unavoidable sources. In particular, the flexibility researchers have when designing studies and analyzing data adds additional

Drawing upon recent developments in structural equation modeling, the current study presents an analytical framework for addressing research questions in which, rather than focusing on means, it is intraindividual (or intragroup) variability that is of direct research interest. Beyond merely serving as an alternative to existing multilevel modeling approaches, this framework allows for extensions to

Careful consideration of the tradeoff between Type I and Type II error rates when designing experiments is critical for maximizing statistical decision accuracy. Typically, Type I error rates (e.g., .05) are significantly lower than Type II error rates (e.g., .20 for .80 power) in psychological science. Further, positive findings (true effects and Type I errors) are more likely to be the focus of replication

Statistical network models describing multivariate dependency structures in psychological data have gained increasing popularity. Such comparably novel statistical techniques require specific guidelines to make them accessible to the research community. So far, researchers have provided tutorials guiding the estimation of networks and their accuracy. However, there is currently little guidance in determining

An evaluation of a difference between effect sizes from two dependent variables in a single study is likely to be based on differences between standard scores if raw scores on those variables are not scaled in comparable units of measurement. The standardization used for this purpose is usually sample-based rather than population-based, but the consequences of this distinction for the construction

The last 25 years have shown a steady increase in attention for the Bayes factor as a tool for hypothesis evaluation and model selection. The present review highlights the potential of the Bayes factor in psychological research. We discuss six types of applications: Bayesian evaluation of point null, interval, and informative hypotheses, Bayesian evidence synthesis, Bayesian variable selection and

The central limit theorem (CLT) is one of the most important theorems in statistics, and it is often introduced to social sciences researchers in an introductory statistics course. However, the recent replication crisis in the social sciences prompts us to investigate just how common certain misconceptions of statistical concepts are. The main purposes of this article are to investigate the misconceptions

Inferences about hypotheses are ubiquitous in the cognitive sciences. Bayes factors provide one general way to compare different hypotheses by their compatibility with the observed data. Those quantifications can then also be used to choose between hypotheses. While Bayes factors provide an immediate approach to hypothesis testing, they are highly sensitive to details of the data/model assumptions

egression models are ubiquitous in the psychological sciences. The standard practice in reporting and interpreting regression models are to present and interpret coefficient estimates and the associated standard errors, confidence intervals and p-values. However, coefficient estimates have limited inferential utility if the outcome is modeled nonlinearly with respect to the substantively interpreted

Network models like the Ising model are increasingly used in psychological research. In a recent article published in this journal, Brusco et al. (2019) provide a critical assessment of the conditions that underlie the Ising model and the eLasso method that is commonly used to estimate it. In this commentary, we show that their main criticisms are unfounded. First, where Brusco et al. (2019) suggest