Integrative data analysis (IDA) jointly models participant-level data from multiple studies. In psychology, IDA has been conducted using different fixed-effects and multilevel modeling (MLM) approaches. However, evaluations regarding the performance of these models in an IDA context are limited. The goal of this article is to evaluate three fixed-effects models (aggregated vs. disaggregated vs. study-specific

Structural equation models (SEMs) are widely used to handle multiequation systems that involve latent variables, multiple indicators, and measurement error. Maximum likelihood (ML) and diagonally weighted least squares (DWLS) dominate the estimation of SEMs with continuous or categorical endogenous variables, respectively. When a model is correctly specified, ML and DWLS function well. But, in the

hen multiple mediators exist on the causal pathway from treatment to outcome, path analysis prevails for disentangling indirect effects along paths linking possibly several mediators. However, separately evaluating each indirect effect along different posited paths demands stringent assumptions, such as correctly specifying the mediators' causal structure, and no unobserved confounding among the mediators

Estimating causal relations between two or more variables is an important topic in psychology. Establishing a causal relation between two variables can help us in answering that question of why something happens. However, using solely observational data are insufficient to get the complete causal picture. The combination of observational and experimental data may give adequate information to properly

Social scientists have become increasingly interested in using intensive longitudinal methods to study social phenomena that change over time. Many of these phenomena are expected to exhibit cycling fluctuations (e.g., sleep, mood, sexual desire). However, researchers typically employ analytical methods which are unable to model such patterns. We present spectral and cross-spectral analysis as means

Exploratory factor analysis (EFA) is a popular method for elucidating the latent structure of data. Unfortunately, EFA models can sometimes produce improper solutions with nonsensical results. For example, improper EFA solutions can include one or more Heywood cases, where common factors account for 100% or more of an observed variable's variance. To better understand these senseless estimates, we

issing values that are missing not at random (MNAR) can result from a variety of missingness processes. However, two fundamental subtypes of MNAR values can be obtained from the definition of the MNAR mechanism itself. The distinction between them deserves consideration because they have characteristic differences in how they distort relationships in the data. This has implications for the validity

Autoregressive and vector autoregressive models are a driving force in current psychological research. In affect research they are, for instance, frequently used to formalize affective processes and estimate affective dynamics. Discrete-time model variants are most commonly used, but continuous-time formulations are gaining popularity, because they can handle data from longitudinal studies in which

n real data analysis with structural equation modeling, data are unlikely to be exactly normally distributed. If we ignore the non-normality reality, the parameter estimates, standard error estimates, and model fit statistics from normal theory based methods such as maximum likelihood (ML) and normal theory based generalized least squares estimation (GLS) are unreliable. On the other hand, the asymptotically

Traditionally, researchers have used time series and multilevel models to analyze intensive longitudinal data. However, these models do not directly address traits and states which conceptualize the stability and variability implicit in longitudinal research, and they do not explicitly take into account measurement error. An alternative to overcome these drawbacks is to consider structural equation

Monte Carlo simulations are widely used in the social sciences to explore the viability of analytic methods in the face of assumption violations. Simulation results, however, may not be applicable to substantive research applications because they often are conducted under idealized rather than realistic conditions. Shortcomings of simulation design are discussed using linear equations as a case study

Monte Carlo simulations are widely used in the social sciences to explore the viability of analytic methods in the face of assumption violations. Simulation results, however, may not be applicable to substantive research applications because they often are conducted under idealized rather than realistic conditions. Shortcomings of simulation design are discussed using linear equations as a case study

Mixed-worded scales require a more careful reading and answering process than scales with only one type of wording. The present study is about the unintended consequences of using such scales, because typically, not all respondents answer positively and negatively worded items consistently. This population heterogeneity-meaning that there are distinct groups of consistently and inconsistently answering

Mixed-worded scales require a more careful reading and answering process than scales with only one type of wording. The present study is about the unintended consequences of using such scales, because typically, not all respondents answer positively and negatively worded items consistently. This population heterogeneity-meaning that there are distinct groups of consistently and inconsistently answering

Current interrater reliability (IRR) coefficients ignore the nested structure of multilevel observational data, resulting in biased estimates of both subject- and cluster-level IRR. We used generalizability theory to provide a conceptualization and estimation method for IRR of continuous multilevel observational data. We explain how generalizability theory decomposes the variance of multilevel observational

Current interrater reliability (IRR) coefficients ignore the nested structure of multilevel observational data, resulting in biased estimates of both subject- and cluster-level IRR. We used generalizability theory to provide a conceptualization and estimation method for IRR of continuous multilevel observational data. We explain how generalizability theory decomposes the variance of multilevel observational

Structural equation modeling (SEM) of ordinal data is often performed using normal theory maximum likelihood estimation based on the Pearson correlation (cont-ML) or using least squares principles based on the polychoric correlation matrix (cat-LS). While cont-ML ignores the categorical nature of the data, cat-LS assumes underlying multivariate normality. Theoretical results are provided on the validity

Psychology research often focuses on interactions, and this has deep implications for inference from nonrepresentative samples. For the goal of estimating average treatment effects, we propose to fit a model allowing treatment to interact with background variables and then average over the distribution of these variables in the population. This can be seen as an extension of multilevel regression and

Deep learning has revolutionized predictive modeling in topics such as computer vision and natural language processing but is not commonly applied to psychological data. In an effort to bring the benefits of deep learning to psychologists, we provide an overview of deep learning for researchers who have a working knowledge of linear regression. We first discuss several benefits of the deep learning

In a world of big data and computational resources, there has been a growing interest in further validating computational models of decision making by subjecting them to more rigorous constraints. One prominent area of study is model-based cognitive neuroscience, where measures of neural activity are explained and interpreted through the lens of a cognitive model. Although some early work has developed

Latent growth curve models with spline functions are flexible and accessible statistical tools for investigating nonlinear change patterns that exhibit distinct phases of development in manifested variables. Among such models, the bilinear spline growth model (BLSGM) is the most straightforward and intuitive but useful. An existing study has demonstrated that the BLSGM allows the knot (or change-point)

Latent growth curve models with spline functions are flexible and accessible statistical tools for investigating nonlinear change patterns that exhibit distinct phases of development in manifested variables. Among such models, the bilinear spline growth model (BLSGM) is the most straightforward and intuitive but useful. An existing study has demonstrated that the BLSGM allows the knot (or change-point)

Lim and Jahng (2019) recently reported simulations supporting the conclusion that traditional parallel analysis (PA) performs more reliably than do more recent PA versions, particularly in the presence of minor factors acting as population error. With noise factors, correct identification of the number of main factors may, however, mean retaining a noise dimension at the expense of missing a signal

Lim and Jahng (2019) recently reported simulations supporting the conclusion that traditional parallel analysis (PA) performs more reliably than do more recent PA versions, particularly in the presence of minor factors acting as population error. With noise factors, correct identification of the number of main factors may, however, mean retaining a noise dimension at the expense of missing a signal

Temporal complexity refers to qualities of a time series that are emergent, erratic, or not easily described by linear processes. Quantifying temporal complexity within a system is key to understanding the time based dynamics of said system. However, many current methods of complexity quantification are not widely used in psychological research because of their technical difficulty, computational intensity

Regularization methods such as the least absolute shrinkage and selection operator (LASSO) are commonly used in high dimensional data to achieve sparser solutions. Recently, methods such as regularized structural equation modeling (SEM) and penalized likelihood SEM have been proposed, trying to transfer the benefits of regularization to models commonly used in social and behavioral research. These

This article compares two missing data procedures, full information maximum likelihood (FIML) and multiple imputation (MI), to investigate their relative performances in relation to the results from analyses of the original complete data or the hypothetical data available before missingness occurred. By expressing the FIML estimator as a special MI estimator, we predicted the expected patterns of discrepancy

Newly available data streams from experience sampling studies and social media are providing new opportunities to study individuals' dyadic relations. The "one-with-many" (OWM) model (Kenny et al., 2006; Kenny & Winquist, 2001) was specifically constructed for and is used to examine features of multiple dyadic relationships that one set of focal persons (e.g., therapists, physicians) has with others

Psychological research often builds on between-group comparisons of (measurements of) latent variables; for instance, to evaluate cross-cultural differences in neuroticism or mindfulness. A critical assumption in such comparative research is that the same latent variable(s) are measured in exactly the same way across all groups (i.e., measurement invariance). Otherwise, one would be comparing apples

Measures of explained variance, ΔR2 and f,2 are routinely used to evaluate the size of moderation effects. However, they suffer from several drawbacks: (a) Not all the variance components of the outcome variable Y are related to the effect of moderation, and so an effect size with the total variance of Y as the denominator cannot accurately characterize the moderation effect; (b) moderation and interaction

We investigated the reproducibility of the major statistical conclusions drawn in 46 articles published in 2012 in three APA journals. After having identified 232 key statistical claims, we tried to reproduce, for each claim, the test statistic, its degrees of freedom, and the corresponding p value, starting from the raw data that were provided by the authors and closely following the Method section

One of the most important methods that psychological scientists use to understand behavior and cognition is theorizing. Increasingly, theorizing is used to support not only additive hypotheses, but also multiplicative ones. And yet, authors often struggle to provide adequate theoretical justifications for multiplicative hypotheses. In the late 1960s and early 1970s, W. Mischel averred that, in "strong"

We develop a Bayesian group-based trajectory model (GBTM) and extend it to incorporate dual trajectories and Bayesian model averaging for model selection. Our framework lends itself to many of the standard distributions used in GBTMs, including normal, censored normal, binary, and ordered outcomes. On the model selection front, GBTMs require the researcher to specify a functional relationship between

Despite increased attention to the role of statistical power in psychological studies, navigating the process of sample size planning for linear regression designs can be challenging. In particular, it can be difficult to decide upon an appropriate value for the effect size, owing to a variety of factors, including the influence of the correlations among the predictors and between the other predictors

An important task of meta-analysis is to observe, quantify, and explain the heterogeneity across the reported effect sizes of primary studies. A primary issue that challenges this task is the myriad of subtle factors that could have contributed to the observed heterogeneity. We leveraged the recent advances in theoretical machine learning to develop a novel latent mixture-based method for disentangling

A huge portion of what we know about how humans develop, learn, behave, and interact is based on survey data. Researchers use longitudinal growth modeling to understand the development of students on psychological and social-emotional learning constructs across elementary and middle school. In these designs, students are typically administered a consistent set of self-report survey items across multiple

The randomized pretest posttest design is common in psychology, as is the corresponding missing data concern. Although missing data handling has seen advances over the past several decades, effective and practical solutions for handling missing data in randomized pretest posttest designs are lacking, particularly when assumptions of commonly used statistical models are violated. Although analysis of

Congruence hypotheses play a major role in many areas of psychology. They refer to, for example, the consequences of person-environment fit, similarity, or self-other agreement. For example, are people psychologically better adjusted when their self-view is in line with their reputation? A valid statistical approach that can be applied to investigate congruence hypotheses of this kind is quadratic

Given the challenges to the notion of objectivity posed by social psychological research on investigator effects, constructivist and critical epistemological perspectives, and the introduction of qualitative research methods in psychology, the investigators examined how leading methodologists understand the function of objectivity and subjectivity in psychological science. The aim of the study was

Missing data is a common feature of experimental datasets. Standard methods used by psychology researchers to handle missingness rely on unrealistic assumptions, invalidate random assignment procedures, and bias estimates of effect sizes. We describe different classes of missing data typically encountered in experimental datasets, and we discuss how each of them impacts researchers' causal inferences

Longitudinal processes rarely occur in isolation; often the growth curves of 2 or more variables are interdependent. Moreover, growth curves rarely exhibit a constant pattern of change. Many educational and psychological phenomena are comprised of different developmental phases (segments). Bivariate piecewise linear mixed-effects models (BPLMEM) are a useful and flexible statistical framework that

We propose a new sequential hypothesis testing procedure in which data are collected and analyzed in a series of independent segments. As in fixed-sample hypothesis testing and in previous sequential procedures, the overall α level can be set to any desired value. Like other sequential procedures, the independent segments procedure generally requires smaller samples than fixed-sample procedures-often

Research on individual variation has received increased attention. The bulk of the models discussed in psychological research so far, focus mainly on the temporal development of the mean structure. We expand the view on within-person residual variability and present a new model parameterization derived from classic multivariate GARCH models used to predict and forecast volatility in financial time-series

Social influence occurs when an individual's outcome is affected by another individual's actions. Current approaches in psychology that seek to examine social influence have focused on settings where individuals are nested in predefined groups and do not interact across groups. Such study designs permit using standard estimation methods such as multilevel models for estimating treatment effects but

Many approaches in the item response theory (IRT) literature have incorporated response styles to control for potential biases. However, the specific assumptions about response styles are often not made explicit. Having integrated different IRT modeling variants into a superordinate framework, we highlighted assumptions and restrictions of the models (Henninger & Meiser, 2020). In this article, we

The p value is still misinterpreted as the probability that the null hypothesis is true. Even psychologists who correctly understand that p values do not provide this probability may not realize the degree to which p values differ from the probability that the null hypothesis is true. Importantly, previous research on this topic has not addressed the influence of multiple testing, often a reality in

A large variety of item response theory (IRT) modeling approaches aim at measuring and correcting for response styles in rating data. Here, we integrate response style models of the divide-by-total model family into one superordinate framework that parameterizes response styles as person-specific shifts in threshold parameters. This superordinate framework allows us to structure and compare existing

Reports an error in "Do simple slopes follow-up tests lead us astray? Advancements in the visualization and reporting of interactions" by Megan C. Finsaas and Brandon L. Goldstein (Psychological Methods, Advanced Online Publication, Apr 20, 2020, np). In the article, Figure 5 contained an error. The second sentence of the caption of Figure 5 should read: "The left plot depicts the region of significance

The Bayesian statistical framework requires the specification of prior distributions, which reflect predata knowledge about the relative plausibility of different parameter values. As prior distributions influence the results of Bayesian analyses, it is important to specify them with care. Prior elicitation has frequently been proposed as a principled method for deriving prior distributions based on

Randomized response models (RRMs) aim at increasing the validity of measuring sensitive attributes by eliciting more honest responses through anonymity protection of respondents. This anonymity protection is achieved by implementing randomization in the questioning procedure. On the other hand, this randomization increases the sampling variance and, therefore, increases sample size requirements. The

Psychology researchers are rapidly adopting open science practices, yet clear guidelines on how to apply these practices to meta-analysis remain lacking. In this tutorial, we describe why open science is important in the context of meta-analysis in psychology, and suggest how to adopt the 3 main components of open science: preregistration, open materials, and open data. We first describe how to make

Numerous psychological and educational researchers are concerned with the moderating effect of a categorical variable on the relationship between a continuous predictor and a continuous response. Despite the great interest in moderation analyses, there exist few studies that explicitly address the essential issues of effect size measure, power calculation, and sample size determination under heterogeneity

Numerous tutorial publications are available to researchers seeking the procedures needed to analyze longitudinal count response variable data. However, most of the available tutorial publications have drawbacks that limit their usefulness to applied researchers, and to the best of our knowledge, very few publications make both the sample data and the data analysis syntax scripts available to readers

When a person takes alternative forms of the same test across replications of the testing procedure, the test taker's observed scores on the alternative forms are rarely identical. In educational and psychological measurement, inconsistencies in a test taker's scores that are irrelevant to the construct being measured are attributed to errors of measurement. Typically, errors of measurement are summarized

A norm-referenced score expresses the position of an individual test taker in the reference population, thereby enabling a proper interpretation of the test score. Such normed scores are derived from test scores obtained from a sample of the reference population. Typically, multiple reference populations exist for a test, namely when the norm-referenced scores depend on individual characteristic(s)

We studied three models for longitudinal mediation analysis: the autoregressive mediation model (AMM) using composite scores (uncorrected composite model, UCM), AMM using composite scores with correction for measurement error (corrected composite model, CCM), and AMM using latent variables with multiple indicators (latent variable model, LVM). Under the condition of unidimensional measurement model

Randomized control trials (RCTs) often use clustered designs, where intact clusters (such as classroom, schools, or treatment centers) are randomly assigned to treatment and control conditions. Hierarchical linear models (HLMs) are used almost universally to estimate the effects in such experiments. While study designs that utilize intact clusters have many potential advantages, there is little guidance

Mediation analysis is an important approach for investigating causal pathways. One approach used in mediation analysis is the test of an indirect effect, which seeks to measure how the effect of an independent variable impacts an outcome variable through 1 or more mediators. However, in many situations the proposed tests of indirect effects, including popular confidence interval-based methods, tend

Science depends on trustworthy evidence. Thus, a biased scientific record is of questionable value because it impedes scientific progress, and the public receives advice on the basis of unreliable evidence that has the potential to have far-reaching detrimental consequences. Meta-analysis is a technique that can be used to summarize research evidence. However, meta-analytic effect size estimates may

Meta-analytic structural equation modeling (MASEM) is an increasingly popular meta-analytic technique that combines the strengths of meta-analysis and structural equation modeling. MASEM facilitates the evaluation of complete theoretical models (e.g., path models or factor analytic models), accounts for sampling covariance between effect sizes, and provides measures of overall fit of the hypothesized