Abstract
The gold standard for modeling multiple indicator measurement data is confirmatory factor analysis (CFA), which has many statistical advantages over traditional exploratory factor analysis (EFA). In most CFA applications, items are assumed to be pure indicators of the construct they intend to measure. However, despite our best efforts, this is often not the case. Cross-loadings incorrectly set to zero can only be expressed through the correlations between the factors, leading to biased factor correlations and to biased structural (regression) parameter estimates. This article introduces a third approach, which has emerged in the psychometric literature, viz., unrestricted factor analysis (UFA). UFA borrows strengths from both traditional EFA and CFA. In simulation studies, we show that ignoring cross-loadings even as low as .2 can substantially bias factor correlations when CFA is used and that even the commonly used guideline RMSEA ≤ .05 may be too lenient to guard against non-negligible bias in factor correlations in CFA. Next, we present two empirical applications using Schwartz’s value theory, and electronic service quality. In the first case, UFA leads to much better model fit and more plausible regression estimates. In the second case, the difference is less dramatic but nevertheless, UFA provides richer results. We provide recommendations on when to use UFA vs. CFA.
Similar content being viewed by others
Notes
Recall that in regression/SEM analysis, β = R−1r, where β is the vector of standardized regression coefficients, R is the correlation matrix of the independent variables and r is the vector of correlations between each independent variable and the dependent variable. Bias in R leads to a different estimate of its inverse (R−1), and hence to bias in the regression coefficients used in theory testing. Standard errors of the regression coefficients are also biased because they involve the square multiple correlation of each predictor i with the other predictors, \({R}_{i}^{2}\):
SEβi = \(\sqrt{(1-{R}_{Y}^{2})/(N-k-1)}\) * \(\sqrt{1/(1-{R}_{i}^{2})}\) where \({R}_{Y}^{2}\) is the explained variance in the dependent variable, N is the sample size, and k is the number of predictors (Cohen et al., 2003, p. 86).
These identification constraints are automatically implemented by Mplus. The researcher can review them with the command TECH1 on the OUTPUT line.
For ease of exposition, we describe the estimation of UFA models as a two-step process as it is the approach used in traditional EFA. In fact, the rotated solution can be estimated directly (Browne 2001), and whether a one-step or two-step estimation approach is used, depends on computational/programming convenience.
Because we use MLR, we use a modified chi-square difference test (Satorra and Bentler 2001). A convenient program to do the calculations is available on https://www.thestatisticalmind.com/calculators/SBChiSquareDifferenceTest.htm.
By June 2022, the article had garnered nearly 6,000 Google Scholar citations.
References
Aruoba, S. B., Diebold, F. X., Nalewaik, J., Schorfheide, F., & Song, D. (2016). Improving GDP measurement: A measurement-error perspective. Journal of Econometrics, 191(2), 384–397.
Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling, 16(3), 397–438.
Asparouhov, T., Muthén, B., & Morin, A. J. S. (2015). Bayesian structural equation modeling with cross-loadings and residual covariances. Journal of Management, 41(September), 1561–1577.
Bagozzi, R. P., & Yi, Y. (1989). On the use of structural equation models in experimental designs. Journal of Marketing Research, 26(3), 271–284.
Batra, R., Homer, P. M., & Kahle, L. R. (2001). Values, susceptibility to normative influence, and attribute importance weights: A nomological analysis. Journal of Consumer Psychology, 11(2), 115–128.
Batra, R., Zhang, C.Y., Aydinoğlu, N. Z., & Feinberg, F. M. (2017). Positioning multicountry brands: The impact of variation in cultural values and competitive set. Journal of Marketing Research, 44(6), 914–931.
Baumgartner, H., & Steenkamp, J.-B. (1996). Exploratory consumer buying behavior: Conceptualization and measurement. International Journal of Research in Marketing, 13(2), 121–137.
Bearden, W. O., Netemeyer, R. G., & Teel, J. E. (1989). Measurement of consumer susceptibility to interpersonal influence. Journal of Consumer Research, 15(March), 473–481.
Bollen, K. A. (1989). Structural equations with latent variables, New York: Wiley.
Bollen, K. A., & Schwing, R. C. (1987). Air pollution-mortality models: A demonstration of the effects of random measurement error. Quality and Quantity, 21, 37–48.
Brangule-Vlagsma, K., Pieters, R. G. M., & Wedel, M. (2002). The dynamics of value segments: Modeling framework and empirical illustration. International Journal of Research in Marketing, 19(3), 267–285.
Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). New York: Guilford Press.
Browne, M. W. (1972). Oblique rotation to a partially specified target. British Journal of Mathematical and Statistical Psychology, 25(2), 207–212.
Browne, M. W. (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36(1), 111–150.
Burroughs, J. E., & Rindfleisch, A. (2002). Materialism and well-being: A conflicting values perspective. Journal of Consumer Research, 29(December), 348–369.
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255.
Church, T. A., & Burke, P. J. (1994). Exploratory and confirmatory tests of the big five and tellegen’s three-and four-dimensional models. Journal of Personality and Social Psychology, 66(1), 93–114.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed. α). Hillsdale: Lawrence Erlbaum
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah: Erlbaum.
Cole, D. A., Ciesla, J. A., & Steiger, J. H. (2007). The insidious effects of failing to include design-driven correlated residuals in latent-variable covariance structure analysis. Psychological Methods, 12(4), 381–398.
Croon, M. A. (2002). Using predicted latent scores in general latent structure models. In: G. A. Marcoulides & I. Moustaki (Eds.), Latent variable and latent structure modeling (pp. 195–223). Mahwah: Erlbaum.
Cudeck, R., & O’Dell, L. L. (1994). Applications of standard error estimates in unrestricted factor analysis: Significance tests for factor loadings and correlations. Psychological Bulletin, 115(3), 475–487.
de Jong, M. G., Fox, J.-P., & Steenkamp, J.-B. (2015). Quantifying under- and overreporting in surveys through a dual-questioning-technique design. Journal of Marketing Research, 52(6), 737–753.
de Luca, L. M., Herhausen, D., Troilo, G., & Rossi, A. (2021). How and when do big data investments pay off? The role of marketing affordances and service innovation. Journal of the Academy of Marketing Science, 49(4), 790–810.
Diener, E., Emmons, R. A., Larsen, R., & Griffin, S. (1985). The satisfaction with life scale, Journal of Personality Assessment, 49, 71–75.
Fischer, M., Völckner, F., & Sattler, H. (2010). How important are brands: A cross-category, cross-country study. Journal of Marketing Research, 47(October), 823–839.
Geuens, M., Weijters, B., & De Wulf, K. (2009). A new measure of brand personality. International Journal of Research in Marketing, 26(1), 97–107.
Green, S. B., & Hershberger, S. L. (2000). Correlated errors in true score models and their effect on coefficient alpha. Structural Equation Modeling, 7(2), 251–270.
Greene, W. H. (2003). Econometric analysis (5th ed). Englewood Cliffs: Prentice-Hall.
Guo, J., Marsh, H. W., Parker, P. D., Dicke, T., Lüdtke, O., & Diallo, T. M. O. (2019). A systematic evaluation and comparison between exploratory structural equation modeling and bayesian structural equation modeling. Structural Equation Modeling, 26(4), 529–556.
Hair, J. F., Babin, B. J., Anderson, R. E., & Black, W. C. (2018). Multivariate statistics, Cengage.
Haws, K. L., Dholakia, U. M., & Bearden, W. O. (2010). An assessment of chronic regulatory focus measures. Journal of Marketing Research, 47(October), 967–982.
Hulland, J., Baumgartner, H., & Smith, K. M. (2018). Marketing survey research best practices: Evidence and recommendations from a review of JAMS articles. Journal of the Academy of Marketing Science, 46, 92–108.
Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 183–202.
Jöreskog, K. G. (1978). Structural analysis of covariance and correlation matrices. Psychometrika, 43(4), 443–477.
Jöreskog, K. G., & Goldberger, A. S. (1975). Estimation of a model with multiple indicators and multiple causes of a single latent variable. Journal of the American Statistical Association, 70(351), 631-639.
MacKenzie, S. B. (2001). Opportunities for improving consumer research through latent variable structural equation modeling. Journal of Consumer Research, 28(1), 159–166.
Marsh, H. W., Guo, J., Dicke, T., Parker, P. D., & Craven, R. G. (2020). Confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), and set ESEM: Optimal balance between goodness of fit and parsimony. Multivariate Behavioral Research, 55(1), 102–119.
Marsh, H. W., Hau, K.-T., & Grayson, D. (2005). Goodness of fit in structural equation models. In: A. Maydeu-Olivares & J. J. McArdle (Eds.), Contemporary Psychometrics, (pp. 275–340). Mahwah: Erlbaum.
Marsh, H. W., Lüdtke, O., Nagengast, B., Morin, A. J. S., & von Davier, M. (2013). Why item parcels are (almost) never appropriate: Two wrongs do not make a right—Camouflaging misspecification with item parcels in CFA. Psychological Methods, 18(3), 257–284.
Marsh, H., Nagengast, B., & Morin, A. J. S. (2013). Measurement invariance of big-five factors over the life span: ESEM tests of gender, age, plasticity, maturity, and La Dolce vita effects. Developmental Psychology, 49(6), 1194–1218.
Marsh, H. W., Morin, A. J. S., Parker, P. D., & Kaur, G. (2014). Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis, Annual Review of Clinical Psychology, 85–110.
Maydeu-Olivares, A. (2017). Assessing the size of model misfit in structural equation models. Psychometrika, 82(3), 533–558.
Maydeu-Olivares, A., & Coffman, D. L. (2006). Random intercept item factor analysis. Psychological Methods, 11(4), 344–362.
McDonald, R. P. (1985). Factor analysis and related methods, Hillsdale: Erlbaum.
McDonald, R. P. (1999). Test theory: A unified approach, Mahwah: Erlbaum.
Morin, A. J. S., Arens, A. K., & Marsh, H. W. (2016). A bifactor exploratory structural equation modeling framework for the identification of distinct sources of construct-relevant psychometric multidimensionality. Structural Equation Modeling, 23(1), 116–139.
Morin A. J. S., Marsh, H. W., & Nagengast, B. (2013). Exploratory structural equation modeling: An introduction. In: G. R. Hancock & R. O. Mueller (Eds.), Structural Equation Modeling: A Second Course (2nd ed., pp. 395– 436). Greenwich: IAP.
Moshagen, M. (2012). The model size effect in SEM: Inflated goodness-of-fit statistics are due to the size of the covariance matrix. Structural Equation Modeling, 19(1), 86–98.
Muthén, B., & Asparouhov, T. (2012). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17(3), 313–335.
Muthén, L. K. & Muthén, B. (2017). Mplus 8, Los Angeles: Muthén & Muthén.
Niemand, T., & Mai, R. (2018). Flexible cutoff values for fit indices in the evaluation of structural equation models. Journal of the Academy of Marketing Science, 46(6), 1148–1172.
Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York, New York: McGraw-Hill.
Parasuraman, A., Zeithaml, V. A., & Malhotra, A. (2005). E-S-QUAL: A multiple-item scale for assessing electronic service quality. Journal of Service Research, 7(3), 213–233.
Paul, M., Hennig-Thurau, T., Gremler, D. D., Gwinner, K. P., & Wiertz, C. (2009). Toward a theory of repeat purchase drivers for consumer services. Journal of the Academy of Marketing Science, 37(2), 215–237.
Pavlov, G., Shi, D., & Maydeu-Olivares, A. (2020). Chi-square difference tests for comparing nested models: An evaluation with non-normal data. Structural Equation Modeling, 27(6), 908–917.
Pedhazur, E. J. (1982). Multiple regression in behavioral research, (2nd ed.). New York: Holt, Rinehart and Winston.
Richins, M. L. (2004). The material values scale: Measurement properties and development of a short form. Journal of Consumer Research, 31(1), 209–219.
Richins, M. L., & Dawson, S. (1992). A consumer values orientation for materialism and its measurement: Scale development and validation. Journal of Consumer Research, 19(3), 303–316.
Roth, M. S. (1995). The effects of culture and socioeconomics on the performance of global brand image strategies. Journal of Marketing Research, 32(May), 163–175.
Ruvio, A., Somer, E., & Rindfleisch, A. (2014). When bad gets worse: The amplifying effect of materialism on traumatic stress and maladaptive consumption. Journal of the Academy of Marketing Science, 42(1), 90–101.
Ryan, L., & Dziurawiec, S. (2001). Materialism and its relationship to life satisfaction. Social Indicators Research, 55(2), 185–197.
Satorra, A. & Bentler, P. M. (1994). Corrections to test statistics and standard errors in covariance structure analysis, In: A. von Eye and C.C. Clogg (Eds.), Latent Variables Analysis: Applications for Developmental Research (pp. 399–419). Thousand Oaks: Sage.
Satorra, A., & Bentler, P. M. (2001). A scaled difference Chi-square test statistic for moment structure analysis. Psychometrika, 66(4), 507–514.
Schwartz, S. H. (1992). Universals in the content and structure of values: theoretical advances and empirical tests in 20 Countries. In M Zanna (Ed.), Advances in Experimental Social Psychology (vol. 25, pp. 1–65). Orlando: Academic Press.
Schwartz, S. H., & Boehnke, K. (2004). Evaluating the structure of human values with confirmatory factor analysis. Journal of Research in Personality, 38, 230–255.
Schwartz, S. H., et al. (2012). Refining the theory of basic individual values. Journal of Personality and Social Psychology, 103(4), 663–688.
Schwartz, S. H., & Sagiv, L. (1995). Identifying culture-specifics in the content and structure of values. Journal of Cross-Cultural Psychology, 26(January), 92–116.
Shepherd, S., Chartrand, T. L., & Fitzsimons, G. J. (2015). When brands reflect our ideal world: The values and brand preferences of consumers who support versus reject society’s dominant ideology. Journal of Consumer Research, 42(1), 76–92.
Shi, D., Maydeu-Olivares, A., & DiStefano, C. (2018). The relationship between the standardized root mean square residual and model misspecification in factor analysis models. Multivariate Behavioral Research, 53(5), 676–694.
Shi, D., Lee, T., & Maydeu-Olivares, A. (2019). Understanding the model size effect on SEM fit indices. Educational and Psychological Measurement, 79(2), 310–334.
Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of cronbach’s alpha. Psychometrika, 74(March), 107–120.
Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal, and structural equation models. CRC Press.
Steenkamp, J.-B., & Baumgartner, H. (1992). The role of optimum stimulation level in exploratory consumer behavior. Journal of Consumer Research, 19(3), 434–448.
Steenkamp, J.-B., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25(1), 78–90.
Steenkamp, J.-B. & Baumgartner, H. (2000). On the use of structural equation models in marketing modeling, International Journal of Research in Marketing, 17 (June-September), 195–202.
Steenkamp, J.-B., & Burgess, S. M. (2002). Optimum stimulation level and exploratory consumer behavior in an emerging consumer market. International Journal of Research in Marketing, 19(2), 131–150.
Steenkamp, J.-B., & de Jong, M. G. (2010). A global investigation into the constellation of consumer attitudes toward global and local products. Journal of Marketing, 74(November), 18–40.
Steenkamp, J.-B., de Jong, M. G., & Baumgartner, H. (2010). Socially desirable response tendencies in survey research. Journal of Marketing Research, 47(April), 199–214.
Steenkamp, J.-B., & Geyskens, I. (2006). How country characteristics affect the perceived value of web sites. Journal of Marketing, 70(July), 136–150.
Steenkamp, J.-B., & Maydeu-Olivares, A. (2015). Stability and change in consumer traits: evidence from a twelve-year longitudinal study, 2002–2013. Journal of Marketing Research, 52(June), 287–308.
Steenkamp, J.-B., & Maydeu-Olivares, A. (2021). An updated paradigm for evaluating measurement invariance incorporating common method variance and its assessment. Journal of the Academy of Marketing Science, 49(1), 5–29.
Steenkamp, J.-B., ter Hofstede, F., & Wedel, M. (1999). A cross-national investigation into the individual and national-cultural antecedents of consumer innovativeness. Journal of Marketing, 63(April), 55–69.
Steenkamp, J.-B., van Heerde, H., & Geyskens, I. (2010). What makes consumers willing to pay a price premium for national brands over private labels? Journal of Marketing Research, 47(December), 1011–1024.
Steenkamp, J.-B., & Wedel, M. (1991). Segmenting retail markets on store image using a consumer-based methodology. Journal of Retailing, 67(3), 300–320.
Swenson, M. J., & Herche, J. (1994). Social values and salesperson performance: An empirical examination. Journal of the Academy of Marketing Science, 22(3), 283–289.
Tabachnick, B. & Fidell, L. (2018). Using multivariate statistics (5th ed.). Boston: Pearson.
Thomson, M., MacInnis, D. J., & Whan Park, C. (2005). The ties that bind: Measuring the strength of consumers’ emotional attachments to brands. Journal of Consumer Psychology, 15(1), 77–91.
Torelli, C. J., Özsomer, A., Carvalho, S. W., Keh, H. T., & Maehle, N. (2012). Brand concepts as representations of human values: Do cultural congruity and compatibility between values matter? Journal of Marketing, 76(4), 92–108.
Unnava, V., & Aravindakshan, A. (2021). How does consumer engagement evolve when brands post across multiple social media? Journal of the Academy of Marketing Science, 49(5), 864–881.
Winterich, K. P., & Zhang, Y. (2014). Accepting inequality deters responsibility: how power distance decreases charitable behavior. Journal of Consumer Research, 41(2), 274–293.
Xie, C., Bagozzi, R. P., & Grønhaug, K. (2015). The role of moral emotions and individual differences in consumer responses to corporate green and non-green actions. Journal of the Academy of Marketing Science, 43(3), 333–356.
Ximénez, C., Maydeu-Olivares, A., Shi, D., & Revuelta, J. (2022). Assessing cutoff values of SEM fit indices: Advantages of the unbiased SRMR index and its cutoff criterion based on communality. Structural Equation Modeling, 29(3), 368–380.
Acknowledgements
We thank the editor and the reviewers for their constructive comments, AiMark for providing the Schwartz Value Survey data and Arvind Malhotra for providing the E-S-QUAL data. The participation of Alberto Maydeu-Olivares was supported, in part, by the Research Center for Child Well-Being (NIGMS P20GM130420), grant PID2020-119755GB-I00 funded by AEI 10.13039/501100011033, and AGAUR grant 1237SGR2017. The cross-national data for the Schwartz Value Survey as well as Mplus and R input files for both empirical applications can be found on: https://www.dropbox.com/sh/tg6573rsx27o21c/AAApQpxdvtZwWoKgosPVUVu-a?dl=0.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
John Hulland served as editor for this article.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Steenkamp, JB.E.M., Maydeu-Olivares, A. Unrestricted factor analysis: A powerful alternative to confirmatory factor analysis. J. of the Acad. Mark. Sci. 51, 86–113 (2023). https://doi.org/10.1007/s11747-022-00888-1
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11747-022-00888-1