Cross-lagged models are by far the most commonly used method to test the prospective effect of one construct on another, yet there are no guidelines for interpreting the size of cross-lagged effects. This research aims to establish empirical benchmarks for cross-lagged effects, focusing on the cross-lagged panel model (CLPM) and the random intercept cross-lagged panel model (RI-CLPM). We drew a quasirepresentative

This note contains a corrective and a generalization of results by Borsboom et al. (2008), based on Heesen and Romeijn (2019). It highlights the relevance of insights from psychometrics beyond the context of psychological testing.

Psychologists have become increasingly interested in the geographical organization of psychological phenomena. Such studies typically seek to identify geographical variation in psychological characteristics and examine the causes and consequences of that variation. Geo-psychological research offers unique advantages, such as a wide variety of easily obtainable behavioral outcomes. However, studies

Researchers are usually interested in examining the impact of covariates when separating heterogeneous samples into latent classes that are more homogeneous. The majority of theoretical and empirical studies with such aims have focused on identifying covariates as predictors of class membership in the structural equation modeling framework. In other words, the covariates only indirectly affect the

A scale to measure a psychological construct is subject to measurement error. When meta-analyzing correlations obtained from scale scores, many researchers recommend correcting for measurement error. I considered three caveats when correcting for measurement error in meta-analysis of correlations: (a) the distribution of true scores can be non-normal, resulting in violation of the normality assumption

When randomized control trials are not available, regression discontinuity (RD) designs are a viable quasi-experimental method shown to be capable of producing causal estimates of how a program or intervention affects an outcome. While the RD design and many related methodological innovations came from the field of psychology, RDs are underutilized among psychologists even though many interventions

Meta-analysis is an important quantitative tool for cumulative science, but its application is frustrated by publication bias. In order to test and adjust for publication bias, we extend model-averaged Bayesian meta-analysis with selection models. The resulting robust Bayesian meta-analysis (RoBMA) methodology does not require all-or-none decisions about the presence of publication bias, can quantify

Measurement invariance—the notion that the measurement properties of a scale are equal across groups, contexts, or time—is an important assumption underlying much of psychology research. The traditional approach for evaluating measurement invariance is to fit a series of nested measurement models using multiple-group confirmatory factor analyses. However, traditional approaches are strict, vary across

In the social sciences, measurement scales often consist of ordinal items and are commonly analyzed using factor analysis. Either data are treated as continuous, or a discretization framework is imposed in order to take the ordinal scale properly into account. Correlational analysis is central in both approaches, and we review recent theory on correlations obtained from ordinal data. To ensure appropriate

Regression models with interaction terms are common models for moderating relationships. When effects of several predictors from one group—for example, genetic variables—are potentially moderated by several predictors from another—for example, environmental variables—many interaction terms result. This complicates model interpretation, especially when coefficient signs point in different directions

When multiple hypothesis tests are conducted, the familywise Type I error probability correspondingly increases. Various multiple test procedures (MTPs) have been developed, which generally aim to control the familywise Type I error rate at the desired level. However, although multiplicity is frequently discussed in the ANOVA literature and MTPs are correspondingly employed, the issue has received

Growth mixture models (GMMs) are a popular method to identify latent classes of growth trajectories. One shortcoming of GMMs is nonconvergence, which often leads researchers to apply covariance equality constraints to simplify estimation, though this may be a dubious assumption. Alternative model specifications have been proposed to reduce nonconvergence without imposing covariance equality constraints

Single case experimental designs (SCEDs) are used to test treatment effects in a wide range of fields and consist of repeated measurements for a single case throughout one or more baseline phases and throughout one or more treatment phases. Recently, mediation analysis has been applied to SCEDs. Mediation analysis decomposes the total treatment-outcome effect into a direct and indirect effect, and

We introduce a new meta-analysis estimator, the weighted and iterated least squares (WILS), that greatly reduces publication selection bias (PSB) when selective reporting for statistical significance (SSS) is present. WILS is the simple weighted average that has smaller bias and rates of false positives than conventional meta-analysis estimators, the unrestricted weighted least squares (UWLS), and

Instead of using manifest proxies for a latent outcome or latent covariates in a causal effect analysis, the R package EffectLiteR facilitates a direct integration of latent variables based on structural equation models (SEM). The corresponding framework considers latent interactions and provides various effect estimates for evaluating the differential effectiveness of treatments. In addition, a user-friendly

Complexity science and systems thinking are increasingly recognized as relevant paradigms for studying systems where biology, psychology, and socioenvironmental factors interact. The application of systems thinking, however, often stops at developing a conceptual model that visualizes the mapping of causal links within a system, e.g., a causal loop diagram (CLD). While this is an important contribution

Publication bias poses a challenge for accurately synthesizing research findings using meta-analysis. A number of statistical methods have been developed to combat this problem by adjusting the meta-analytic estimates. Previous studies tended to apply these methods without regard to optimal conditions for each method’s performance. The present study sought to estimate the typical effect size attenuation

Various theoretical accounts suggest that within-person effects relating to everyday experiences (assessed, e.g., via experience sampling studies or daily diary studies) are a central element for understanding between-person differences in future outcomes. In this regard, it is often assumed that the within-person effect of a time-varying predictor X on a time-varying mediator M contributes to the

Model comparison is the cornerstone of theoretical progress in psychological research. Common practice overwhelmingly relies on tools that evaluate competing models by balancing in-sample descriptive adequacy against model flexibility, with modern approaches advocating the use of marginal likelihood for hierarchical cognitive models. Cross-validation is another popular approach but its implementation

Eye-tracking has emerged as a popular method for empirical studies of cognitive processes across multiple substantive research areas. Eye-tracking systems are capable of automatically generating fixation-location data over time at high temporal resolution. Often, the researcher obtains a binary measure of whether or not, at each point in time, the participant is fixating on a critical interest area

Various theoretical accounts suggest that within-person effects relating to everyday experiences (assessed, e.g., via experience sampling studies or daily diary studies) are a central element for understanding between-person differences in future outcomes. In this regard, it is often assumed that the within-person effect of a time-varying predictor X on a time-varying mediator M contributes to the

Network psychometrics is undergoing a time of methodological reflection. In part, this was spurred by the revelation that ℓ₁-regularization does not reduce spurious associations in partial correlation networks. In this work, we address another motivation for the widespread use of regularized estimation: the thought that it is needed to mitigate overfitting. We first clarify important aspects of overfitting

Bayesian t tests have become increasingly popular alternatives to null-hypothesis significance testing (NHST) in psychological research. In contrast to NHST, they allow for the quantification of evidence in favor of the null hypothesis and for optional stopping. A major drawback of Bayesian t tests, however, is that error probabilities of statistical decisions remain uncontrolled. Previous approaches

Bayesian hypothesis testing procedures have gained increased acceptance in recent years. A key advantage that Bayesian tests have over classical testing procedures is their potential to quantify information in support of true null hypotheses. Ironically, default implementations of Bayesian tests prevent the accumulation of strong evidence in favor of true null hypotheses because associated default

Researchers across varied fields increasingly are collecting and analyzing intensive longitudinal data (ILD) to examine processes across time at the individual level. Two types of relations are typically examined: lagged and contemporaneous. Lagged relations capture how variables at a prior time point can be used to explain variance in variables at a later time point. These are always modeled using

Multimethod measurement designs with planned missing data (MMM-PMD) aim at combining cheap proxy methods (e.g., self-reports) with an expensive gold standard method (e.g., biomarker) in order to improve the cost-efficiency of research designs. This article presents a comprehensive simulation study investigating whether accurate factor scores with trustworthy confidence or credible intervals for the

Statistical network models describing multivariate dependency structures in psychological data have gained increasing popularity. Such comparably novel statistical techniques require specific guidelines to make them accessible to the research community. So far, researchers have provided tutorials guiding the estimation of networks and their accuracy. However, there is currently little guidance in determining

Network approaches to psychometric constructs, in which constructs are modeled in terms of interactions between their constituent factors, have rapidly gained popularity in psychology. Applications of such network approaches to various psychological constructs have recently moved from a descriptive stance, in which the goal is to estimate the network structure that pertains to a construct, to a more

Network approaches to psychometric constructs, in which constructs are modeled in terms of interactions between their constituent factors, have rapidly gained popularity in psychology. Applications of such network approaches to various psychological constructs have recently moved from a descriptive stance, in which the goal is to estimate the network structure that pertains to a construct, to a more

Concerns about replication failures can be partially recast as concerns about excessive heterogeneity in research results. Although this heterogeneity is an inherent part of science (e.g., sampling variability; studying different conditions), not all heterogeneity results from unavoidable sources. In particular, the flexibility researchers have when designing studies and analyzing data adds additional

Drawing upon recent developments in structural equation modeling, the current study presents an analytical framework for addressing research questions in which, rather than focusing on means, it is intraindividual (or intragroup) variability that is of direct research interest. Beyond merely serving as an alternative to existing multilevel modeling approaches, this framework allows for extensions to

Careful consideration of the tradeoff between Type I and Type II error rates when designing experiments is critical for maximizing statistical decision accuracy. Typically, Type I error rates (e.g., .05) are significantly lower than Type II error rates (e.g., .20 for .80 power) in psychological science. Further, positive findings (true effects and Type I errors) are more likely to be the focus of replication

Statistical network models describing multivariate dependency structures in psychological data have gained increasing popularity. Such comparably novel statistical techniques require specific guidelines to make them accessible to the research community. So far, researchers have provided tutorials guiding the estimation of networks and their accuracy. However, there is currently little guidance in determining

An evaluation of a difference between effect sizes from two dependent variables in a single study is likely to be based on differences between standard scores if raw scores on those variables are not scaled in comparable units of measurement. The standardization used for this purpose is usually sample-based rather than population-based, but the consequences of this distinction for the construction

The last 25 years have shown a steady increase in attention for the Bayes factor as a tool for hypothesis evaluation and model selection. The present review highlights the potential of the Bayes factor in psychological research. We discuss six types of applications: Bayesian evaluation of point null, interval, and informative hypotheses, Bayesian evidence synthesis, Bayesian variable selection and

The central limit theorem (CLT) is one of the most important theorems in statistics, and it is often introduced to social sciences researchers in an introductory statistics course. However, the recent replication crisis in the social sciences prompts us to investigate just how common certain misconceptions of statistical concepts are. The main purposes of this article are to investigate the misconceptions

Inferences about hypotheses are ubiquitous in the cognitive sciences. Bayes factors provide one general way to compare different hypotheses by their compatibility with the observed data. Those quantifications can then also be used to choose between hypotheses. While Bayes factors provide an immediate approach to hypothesis testing, they are highly sensitive to details of the data/model assumptions

egression models are ubiquitous in the psychological sciences. The standard practice in reporting and interpreting regression models are to present and interpret coefficient estimates and the associated standard errors, confidence intervals and p-values. However, coefficient estimates have limited inferential utility if the outcome is modeled nonlinearly with respect to the substantively interpreted

Network models like the Ising model are increasingly used in psychological research. In a recent article published in this journal, Brusco et al. (2019) provide a critical assessment of the conditions that underlie the Ising model and the eLasso method that is commonly used to estimate it. In this commentary, we show that their main criticisms are unfounded. First, where Brusco et al. (2019) suggest

Psychological researchers often use standard linear regression to identify relevant predictors of an outcome of interest, but challenges emerge with incomplete data and growing numbers of candidate predictors. Regularization methods like the LASSO can reduce the risk of overfitting, increase model interpretability, and improve prediction in future samples; however, handling missing data when using

Bayesian Networks are probabilistic graphical models that represent conditional independence relationships among variables as a directed acyclic graph (DAG), where edges can be interpreted as causal effects connecting one causal symptom to an effect symptom. These models can help overcome one of the key limitations of partial correlation networks whose edges are undirected. This tutorial aims to introduce

This article presents techniques for dealing with a form of dependency in data arising when numerical data sum to a constant for individual cases, that is, “compositional” or “ipsative” data. Examples are percentages that sum to 100, and hours in a day that sum to 24. Ipsative scales fell out of fashion in psychology during the 1960s and 1970s due to a lack of methods for analyzing them. However, ipsative

The topic of replicability has recently captivated the emerging field of network psychometrics. Although methodological practice (e.g., p-hacking) has been identified as a root cause of unreliable research findings in psychological science, the statistical model itself has come under attack in the partial correlation network literature. In a motivating example, I first describe how sampling variability

This article proposes a summary-statistics-based power analysis—a practical method for conducting power analysis for mixed-effects modeling with two-level nested data (for both binary and continuous predictors), complementing the existing formula-based and simulation-based methods. The proposed method bases its logic on conditional equivalence of the summary-statistics approach and mixed-effects modeling

Controversy over which reliability estimators should be used persists due to a lack of knowledge about their accuracy. Simulation is an effective tool to obtain an answer, but existing simulation studies yield contradictory results regarding which reliability estimators are the best. The causes of these inconsistent conclusions have yet to be discussed. This study reanalyzes existing studies to understand

Latent variable models (LVMs) are incredibly flexible tools that allow users to address research questions they might otherwise never be able to answer (McDonald, 2013). However, one major limitation of LVMs is evaluating model fit. There is no universal consensus about how to evaluate model fit, either globally or locally. Part of the reason evaluating these models is difficult is because fit is typically

The Social Relations Model (SRM) is a conceptual and mathematical model of interpersonal responses in dyads. The SRM permits estimation of responses of one to many (the actor effect) and the responses of many to the one (the partner effect) at the individual level of analysis. The SRM also permits estimation of the unique responses of actors and partners in specific dyadic arrangements (the relationship

The matching paradigm can take a number of forms and has been used in many areas of psychology. When participants are asked to match or order sets of objects, researchers must correctly account for the number of matches expected purely by chance. Not accounting for the expected chance matches can lead to incorrectly drawing conclusions based on one's data. This study demonstrated that the z test can

Failure of replication attempts in experimental psychology might extend beyond p-hacking, publication bias or hidden moderators; reductions in experimental power can be caused by violations of fidelity to a set of experimental protocols. In this article, we run a series of simulations to systematically explore how manipulating fidelity influences effect size. We find statistical patterns that mimic

Just-in-time adaptive interventions (JITAIs) are time-varying adaptive interventions that use frequent opportunities for the intervention to be adapted—weekly, daily, or even many times a day. The microrandomized trial (MRT) has emerged for use in informing the construction of JITAIs. MRTs can be used to address research questions about whether and under what circumstances JITAI components are effective

The Likert item response format for items is almost ubiquitous in the social sciences and has particular virtues regarding the relative simplicity of item-generation and the efficiency for coding responses. However, in this article, we critique this very common item format, focusing on its affordance for interpretation in terms of internal structure validity evidence. We suggest an alternative, the

The full-information maximum likelihood (FIML) is a popular estimation method for missing data in structural equation modeling (SEM). However, previous research has shown that SEM approximate fit indices (AFIs) such as the root mean square error of approximation (RMSEA) and the comparative fit index (CFI) can be distorted relative to their complete data counterparts when they are computed following

We introduce a new method for estimating the degree of nonadditivity in a one-facet generalizability theory design. One-facet G-theory designs have only one observation per cell, such as persons answering items in a test, and assume that there is no interaction between facets. When there is interaction, the model becomes nonadditive, and G-theory variance estimates and reliability coefficients are

Measurement invariance research has focused on identifying biases in test indicators measuring a latent trait across two or more groups. However, relatively little attention has been devoted to the practical implications of noninvariance. An important question is whether noninvariance in indicators or items results in differences in observed composite scores across groups. The current study introduces

In high stakes assessments of personality and similar attributes, test takers may engage in impression management (aka faking). This article proposes to consider responses of every test taker as a potential mixture of “real” (or retrieved) answers to questions, and “ideal” answers intended to create a desired impression, with each type of response characterized by its own distribution and factor structure

Many real-world systems can exhibit tipping points and multiple stable states, creating the potential for sudden and difficult to reverse transitions into a less desirable regime. The theory of dynamical systems points to the existence of generic early warning signals that may precede these so-called critical transitions. Recently, psychologists have begun to conceptualize mental disorders such as

Recent times have seen a call for personalized psychotherapy and tailored communication during treatment, leading to the necessity to model the complex dynamics of mental disorders in a single subject. To this aim, time-series data in one patient can be collected through ecological momentary assessment and analyzed with the graphical vector autoregressive model, estimating temporal and contemporaneous

Confirmatory factor analysis (CFA) and its bifactor models are popular in empirical investigations of the factor structure of psychological constructs. CFA offers straightforward hypothesis testing but has notable pitfalls, such as the imposition of strict assumptions (i.e., simple structure) that obscure unmodeled complexity. Due to the limitations of bifactor CFAs, they have yielded anomalous results

In exploratory factor analysis, factor rotation algorithms can converge to local solutions (i.e., local minima) when they are initiated from different starting points. To better understand this problem, we performed three studies that investigated the prevalence and correlates of local solutions with five factor rotation algorithms: varimax, oblimin, entropy, and geomin (orthogonal and oblique). In

Criterion-related profile analysis (CPA) is a least squares linear regression technique for identifying a criterion-related pattern (CRP) among predictor variables and for quantifying the variance accounted for by the pattern. A CRP is a pattern, described by a vector of contrast coefficients, such that predictor profiles with higher similarity to the pattern have higher expected criterion scores.