Skip to main content
Log in

Unrestricted factor analysis: A powerful alternative to confirmatory factor analysis

  • Methodology
  • Published:
Journal of the Academy of Marketing Science Aims and scope Submit manuscript

Abstract

The gold standard for modeling multiple indicator measurement data is confirmatory factor analysis (CFA), which has many statistical advantages over traditional exploratory factor analysis (EFA). In most CFA applications, items are assumed to be pure indicators of the construct they intend to measure. However, despite our best efforts, this is often not the case. Cross-loadings incorrectly set to zero can only be expressed through the correlations between the factors, leading to biased factor correlations and to biased structural (regression) parameter estimates. This article introduces a third approach, which has emerged in the psychometric literature, viz., unrestricted factor analysis (UFA). UFA borrows strengths from both traditional EFA and CFA. In simulation studies, we show that ignoring cross-loadings even as low as .2 can substantially bias factor correlations when CFA is used and that even the commonly used guideline RMSEA ≤ .05 may be too lenient to guard against non-negligible bias in factor correlations in CFA. Next, we present two empirical applications using Schwartz’s value theory, and electronic service quality. In the first case, UFA leads to much better model fit and more plausible regression estimates. In the second case, the difference is less dramatic but nevertheless, UFA provides richer results. We provide recommendations on when to use UFA vs. CFA.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. Recall that in regression/SEM analysis, β = R−1r, where β is the vector of standardized regression coefficients, R is the correlation matrix of the independent variables and r is the vector of correlations between each independent variable and the dependent variable. Bias in R leads to a different estimate of its inverse (R−1), and hence to bias in the regression coefficients used in theory testing. Standard errors of the regression coefficients are also biased because they involve the square multiple correlation of each predictor i with the other predictors, \({R}_{i}^{2}\):

    SEβi = \(\sqrt{(1-{R}_{Y}^{2})/(N-k-1)}\) * \(\sqrt{1/(1-{R}_{i}^{2})}\) where \({R}_{Y}^{2}\) is the explained variance in the dependent variable, N is the sample size, and k is the number of predictors (Cohen et al., 2003, p. 86).

  2. These identification constraints are automatically implemented by Mplus. The researcher can review them with the command TECH1 on the OUTPUT line.

  3. For ease of exposition, we describe the estimation of UFA models as a two-step process as it is the approach used in traditional EFA. In fact, the rotated solution can be estimated directly (Browne 2001), and whether a one-step or two-step estimation approach is used, depends on computational/programming convenience.

  4. Because we use MLR, we use a modified chi-square difference test (Satorra and Bentler 2001). A convenient program to do the calculations is available on https://www.thestatisticalmind.com/calculators/SBChiSquareDifferenceTest.htm.

  5. By June 2022, the article had garnered nearly 6,000 Google Scholar citations.

References

  • Aruoba, S. B., Diebold, F. X., Nalewaik, J., Schorfheide, F., & Song, D. (2016). Improving GDP measurement: A measurement-error perspective. Journal of Econometrics, 191(2), 384–397.

    Article  Google Scholar 

  • Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling, 16(3), 397–438.

    Article  Google Scholar 

  • Asparouhov, T., Muthén, B., & Morin, A. J. S. (2015). Bayesian structural equation modeling with cross-loadings and residual covariances. Journal of Management, 41(September), 1561–1577.

    Article  Google Scholar 

  • Bagozzi, R. P., & Yi, Y. (1989). On the use of structural equation models in experimental designs. Journal of Marketing Research, 26(3), 271–284.

    Article  Google Scholar 

  • Batra, R., Homer, P. M., & Kahle, L. R. (2001). Values, susceptibility to normative influence, and attribute importance weights: A nomological analysis. Journal of Consumer Psychology, 11(2), 115–128.

    Article  Google Scholar 

  • Batra, R., Zhang, C.Y., Aydinoğlu, N. Z., & Feinberg, F. M. (2017). Positioning multicountry brands: The impact of variation in cultural values and competitive set. Journal of Marketing Research, 44(6), 914–931.

    Article  Google Scholar 

  • Baumgartner, H., & Steenkamp, J.-B. (1996). Exploratory consumer buying behavior: Conceptualization and measurement. International Journal of Research in Marketing, 13(2), 121–137.

    Article  Google Scholar 

  • Bearden, W. O., Netemeyer, R. G., & Teel, J. E. (1989). Measurement of consumer susceptibility to interpersonal influence. Journal of Consumer Research, 15(March), 473–481.

    Article  Google Scholar 

  • Bollen, K. A. (1989). Structural equations with latent variables, New York: Wiley.

    Book  Google Scholar 

  • Bollen, K. A., & Schwing, R. C. (1987). Air pollution-mortality models: A demonstration of the effects of random measurement error. Quality and Quantity, 21, 37–48.

    Article  Google Scholar 

  • Brangule-Vlagsma, K., Pieters, R. G. M., & Wedel, M. (2002). The dynamics of value segments: Modeling framework and empirical illustration. International Journal of Research in Marketing, 19(3), 267–285.

    Article  Google Scholar 

  • Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). New York: Guilford Press.

  • Browne, M. W. (1972). Oblique rotation to a partially specified target. British Journal of Mathematical and Statistical Psychology, 25(2), 207–212.

    Article  Google Scholar 

  • Browne, M. W. (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36(1), 111–150.

    Article  Google Scholar 

  • Burroughs, J. E., & Rindfleisch, A. (2002). Materialism and well-being: A conflicting values perspective. Journal of Consumer Research, 29(December), 348–369.

    Article  Google Scholar 

  • Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504.

    Article  Google Scholar 

  • Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255.

    Article  Google Scholar 

  • Church, T. A., & Burke, P. J. (1994). Exploratory and confirmatory tests of the big five and tellegen’s three-and four-dimensional models. Journal of Personality and Social Psychology, 66(1), 93–114.

    Article  Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed. α). Hillsdale: Lawrence Erlbaum

  • Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah: Erlbaum.

  • Cole, D. A., Ciesla, J. A., & Steiger, J. H. (2007). The insidious effects of failing to include design-driven correlated residuals in latent-variable covariance structure analysis. Psychological Methods, 12(4), 381–398.

    Article  Google Scholar 

  • Croon, M. A. (2002). Using predicted latent scores in general latent structure models. In: G. A. Marcoulides & I. Moustaki (Eds.), Latent variable and latent structure modeling (pp. 195–223). Mahwah: Erlbaum.

    Google Scholar 

  • Cudeck, R., & O’Dell, L. L. (1994). Applications of standard error estimates in unrestricted factor analysis: Significance tests for factor loadings and correlations. Psychological Bulletin, 115(3), 475–487.

    Article  Google Scholar 

  • de Jong, M. G., Fox, J.-P., & Steenkamp, J.-B. (2015). Quantifying under- and overreporting in surveys through a dual-questioning-technique design. Journal of Marketing Research, 52(6), 737–753.

    Article  Google Scholar 

  • de Luca, L. M., Herhausen, D., Troilo, G., & Rossi, A. (2021). How and when do big data investments pay off? The role of marketing affordances and service innovation. Journal of the Academy of Marketing Science, 49(4), 790–810.

    Article  Google Scholar 

  • Diener, E., Emmons, R. A., Larsen, R., & Griffin, S. (1985). The satisfaction with life scale, Journal of Personality Assessment, 49, 71–75.

  • Fischer, M., Völckner, F., & Sattler, H. (2010). How important are brands: A cross-category, cross-country study. Journal of Marketing Research, 47(October), 823–839.

    Article  Google Scholar 

  • Geuens, M., Weijters, B., & De Wulf, K. (2009). A new measure of brand personality. International Journal of Research in Marketing, 26(1), 97–107.

    Article  Google Scholar 

  • Green, S. B., & Hershberger, S. L. (2000). Correlated errors in true score models and their effect on coefficient alpha. Structural Equation Modeling, 7(2), 251–270.

    Article  Google Scholar 

  • Greene, W. H. (2003). Econometric analysis (5th ed). Englewood Cliffs: Prentice-Hall.

  • Guo, J., Marsh, H. W., Parker, P. D., Dicke, T., Lüdtke, O., & Diallo, T. M. O. (2019). A systematic evaluation and comparison between exploratory structural equation modeling and bayesian structural equation modeling. Structural Equation Modeling, 26(4), 529–556.

    Article  Google Scholar 

  • Hair, J. F., Babin, B. J., Anderson, R. E., & Black, W. C. (2018). Multivariate statistics, Cengage.

  • Haws, K. L., Dholakia, U. M., & Bearden, W. O. (2010). An assessment of chronic regulatory focus measures. Journal of Marketing Research, 47(October), 967–982.

    Article  Google Scholar 

  • Hulland, J., Baumgartner, H., & Smith, K. M. (2018). Marketing survey research best practices: Evidence and recommendations from a review of JAMS articles. Journal of the Academy of Marketing Science, 46, 92–108.

    Article  Google Scholar 

  • Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 183–202.

    Article  Google Scholar 

  • Jöreskog, K. G. (1978). Structural analysis of covariance and correlation matrices. Psychometrika, 43(4), 443–477.

    Article  Google Scholar 

  • Jöreskog, K. G., & Goldberger, A. S. (1975). Estimation of a model with multiple indicators and multiple causes of a single latent variable. Journal of the American Statistical Association, 70(351), 631-639.

    Article  Google Scholar 

  • MacKenzie, S. B. (2001). Opportunities for improving consumer research through latent variable structural equation modeling. Journal of Consumer Research, 28(1), 159–166.

    Article  Google Scholar 

  • Marsh, H. W., Guo, J., Dicke, T., Parker, P. D., & Craven, R. G. (2020). Confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), and set ESEM: Optimal balance between goodness of fit and parsimony. Multivariate Behavioral Research, 55(1), 102–119.

    Article  Google Scholar 

  • Marsh, H. W., Hau, K.-T., & Grayson, D. (2005). Goodness of fit in structural equation models. In: A. Maydeu-Olivares & J. J. McArdle (Eds.), Contemporary Psychometrics, (pp. 275–340). Mahwah: Erlbaum.

    Google Scholar 

  • Marsh, H. W., Lüdtke, O., Nagengast, B., Morin, A. J. S., & von Davier, M. (2013). Why item parcels are (almost) never appropriate: Two wrongs do not make a right—Camouflaging misspecification with item parcels in CFA. Psychological Methods, 18(3), 257–284.

    Article  Google Scholar 

  • Marsh, H., Nagengast, B., & Morin, A. J. S. (2013). Measurement invariance of big-five factors over the life span: ESEM tests of gender, age, plasticity, maturity, and La Dolce vita effects. Developmental Psychology, 49(6), 1194–1218.

    Article  Google Scholar 

  • Marsh, H. W., Morin, A. J. S., Parker, P. D., & Kaur, G. (2014). Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis, Annual Review of Clinical Psychology, 85–110.

  • Maydeu-Olivares, A. (2017). Assessing the size of model misfit in structural equation models. Psychometrika, 82(3), 533–558.

    Article  Google Scholar 

  • Maydeu-Olivares, A., & Coffman, D. L. (2006). Random intercept item factor analysis. Psychological Methods, 11(4), 344–362.

    Article  Google Scholar 

  • McDonald, R. P. (1985). Factor analysis and related methods, Hillsdale: Erlbaum.

    Google Scholar 

  • McDonald, R. P. (1999). Test theory: A unified approach, Mahwah: Erlbaum.

    Google Scholar 

  • Morin, A. J. S., Arens, A. K., & Marsh, H. W. (2016). A bifactor exploratory structural equation modeling framework for the identification of distinct sources of construct-relevant psychometric multidimensionality. Structural Equation Modeling, 23(1), 116–139.

    Article  Google Scholar 

  • Morin A. J. S., Marsh, H. W., & Nagengast, B. (2013). Exploratory structural equation modeling: An introduction. In: G. R. Hancock & R. O. Mueller (Eds.), Structural Equation Modeling: A Second Course (2nd ed., pp. 395– 436). Greenwich: IAP.

  • Moshagen, M. (2012). The model size effect in SEM: Inflated goodness-of-fit statistics are due to the size of the covariance matrix. Structural Equation Modeling, 19(1), 86–98.

    Article  Google Scholar 

  • Muthén, B., & Asparouhov, T. (2012). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17(3), 313–335.

    Article  Google Scholar 

  • Muthén, L. K. & Muthén, B. (2017). Mplus 8, Los Angeles: Muthén & Muthén.

  • Niemand, T., & Mai, R. (2018). Flexible cutoff values for fit indices in the evaluation of structural equation models. Journal of the Academy of Marketing Science, 46(6), 1148–1172.

    Article  Google Scholar 

  • Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York, New York: McGraw-Hill.

  • Parasuraman, A., Zeithaml, V. A., & Malhotra, A. (2005). E-S-QUAL: A multiple-item scale for assessing electronic service quality. Journal of Service Research, 7(3), 213–233.

    Article  Google Scholar 

  • Paul, M., Hennig-Thurau, T., Gremler, D. D., Gwinner, K. P., & Wiertz, C. (2009). Toward a theory of repeat purchase drivers for consumer services. Journal of the Academy of Marketing Science, 37(2), 215–237.

    Article  Google Scholar 

  • Pavlov, G., Shi, D., & Maydeu-Olivares, A. (2020). Chi-square difference tests for comparing nested models: An evaluation with non-normal data. Structural Equation Modeling, 27(6), 908–917.

    Article  Google Scholar 

  • Pedhazur, E. J. (1982). Multiple regression in behavioral research, (2nd ed.). New York: Holt, Rinehart and Winston.

  • Richins, M. L. (2004). The material values scale: Measurement properties and development of a short form. Journal of Consumer Research, 31(1), 209–219.

    Article  Google Scholar 

  • Richins, M. L., & Dawson, S. (1992). A consumer values orientation for materialism and its measurement: Scale development and validation. Journal of Consumer Research, 19(3), 303–316.

    Article  Google Scholar 

  • Roth, M. S. (1995). The effects of culture and socioeconomics on the performance of global brand image strategies. Journal of Marketing Research, 32(May), 163–175.

    Article  Google Scholar 

  • Ruvio, A., Somer, E., & Rindfleisch, A. (2014). When bad gets worse: The amplifying effect of materialism on traumatic stress and maladaptive consumption. Journal of the Academy of Marketing Science, 42(1), 90–101.

    Article  Google Scholar 

  • Ryan, L., & Dziurawiec, S. (2001). Materialism and its relationship to life satisfaction. Social Indicators Research, 55(2), 185–197.

    Article  Google Scholar 

  • Satorra, A. & Bentler, P. M. (1994). Corrections to test statistics and standard errors in covariance structure analysis, In: A. von Eye and C.C. Clogg (Eds.), Latent Variables Analysis: Applications for Developmental Research (pp. 399–419). Thousand Oaks: Sage.

  • Satorra, A., & Bentler, P. M. (2001). A scaled difference Chi-square test statistic for moment structure analysis. Psychometrika, 66(4), 507–514.

    Article  Google Scholar 

  • Schwartz, S. H. (1992). Universals in the content and structure of values: theoretical advances and empirical tests in 20 Countries. In M Zanna (Ed.), Advances in Experimental Social Psychology (vol. 25, pp. 1–65). Orlando: Academic Press.

  • Schwartz, S. H., & Boehnke, K. (2004). Evaluating the structure of human values with confirmatory factor analysis. Journal of Research in Personality, 38, 230–255.

    Article  Google Scholar 

  • Schwartz, S. H., et al. (2012). Refining the theory of basic individual values. Journal of Personality and Social Psychology, 103(4), 663–688.

    Article  Google Scholar 

  • Schwartz, S. H., & Sagiv, L. (1995). Identifying culture-specifics in the content and structure of values. Journal of Cross-Cultural Psychology, 26(January), 92–116.

    Article  Google Scholar 

  • Shepherd, S., Chartrand, T. L., & Fitzsimons, G. J. (2015). When brands reflect our ideal world: The values and brand preferences of consumers who support versus reject society’s dominant ideology. Journal of Consumer Research, 42(1), 76–92.

    Article  Google Scholar 

  • Shi, D., Maydeu-Olivares, A., & DiStefano, C. (2018). The relationship between the standardized root mean square residual and model misspecification in factor analysis models. Multivariate Behavioral Research, 53(5), 676–694.

    Article  Google Scholar 

  • Shi, D., Lee, T., & Maydeu-Olivares, A. (2019). Understanding the model size effect on SEM fit indices. Educational and Psychological Measurement, 79(2), 310–334.

    Article  Google Scholar 

  • Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of cronbach’s alpha. Psychometrika, 74(March), 107–120.

    Article  Google Scholar 

  • Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal, and structural equation models. CRC Press.

    Book  Google Scholar 

  • Steenkamp, J.-B., & Baumgartner, H. (1992). The role of optimum stimulation level in exploratory consumer behavior. Journal of Consumer Research, 19(3), 434–448.

    Article  Google Scholar 

  • Steenkamp, J.-B., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25(1), 78–90.

    Article  Google Scholar 

  • Steenkamp, J.-B. & Baumgartner, H. (2000). On the use of structural equation models in marketing modeling, International Journal of Research in Marketing, 17 (June-September), 195–202.

  • Steenkamp, J.-B., & Burgess, S. M. (2002). Optimum stimulation level and exploratory consumer behavior in an emerging consumer market. International Journal of Research in Marketing, 19(2), 131–150.

    Article  Google Scholar 

  • Steenkamp, J.-B., & de Jong, M. G. (2010). A global investigation into the constellation of consumer attitudes toward global and local products. Journal of Marketing, 74(November), 18–40.

    Article  Google Scholar 

  • Steenkamp, J.-B., de Jong, M. G., & Baumgartner, H. (2010). Socially desirable response tendencies in survey research. Journal of Marketing Research, 47(April), 199–214.

    Article  Google Scholar 

  • Steenkamp, J.-B., & Geyskens, I. (2006). How country characteristics affect the perceived value of web sites. Journal of Marketing, 70(July), 136–150.

    Article  Google Scholar 

  • Steenkamp, J.-B., & Maydeu-Olivares, A. (2015). Stability and change in consumer traits: evidence from a twelve-year longitudinal study, 2002–2013. Journal of Marketing Research, 52(June), 287–308.

    Article  Google Scholar 

  • Steenkamp, J.-B., & Maydeu-Olivares, A. (2021). An updated paradigm for evaluating measurement invariance incorporating common method variance and its assessment. Journal of the Academy of Marketing Science, 49(1), 5–29.

    Article  Google Scholar 

  • Steenkamp, J.-B., ter Hofstede, F., & Wedel, M. (1999). A cross-national investigation into the individual and national-cultural antecedents of consumer innovativeness. Journal of Marketing, 63(April), 55–69.

    Article  Google Scholar 

  • Steenkamp, J.-B., van Heerde, H., & Geyskens, I. (2010). What makes consumers willing to pay a price premium for national brands over private labels? Journal of Marketing Research, 47(December), 1011–1024.

    Article  Google Scholar 

  • Steenkamp, J.-B., & Wedel, M. (1991). Segmenting retail markets on store image using a consumer-based methodology. Journal of Retailing, 67(3), 300–320.

    Google Scholar 

  • Swenson, M. J., & Herche, J. (1994). Social values and salesperson performance: An empirical examination. Journal of the Academy of Marketing Science, 22(3), 283–289.

    Article  Google Scholar 

  • Tabachnick, B. & Fidell, L. (2018). Using multivariate statistics (5th ed.). Boston: Pearson.

  • Thomson, M., MacInnis, D. J., & Whan Park, C. (2005). The ties that bind: Measuring the strength of consumers’ emotional attachments to brands. Journal of Consumer Psychology, 15(1), 77–91.

    Article  Google Scholar 

  • Torelli, C. J., Özsomer, A., Carvalho, S. W., Keh, H. T., & Maehle, N. (2012). Brand concepts as representations of human values: Do cultural congruity and compatibility between values matter? Journal of Marketing, 76(4), 92–108.

    Article  Google Scholar 

  • Unnava, V., & Aravindakshan, A. (2021). How does consumer engagement evolve when brands post across multiple social media? Journal of the Academy of Marketing Science, 49(5), 864–881.

    Article  Google Scholar 

  • Winterich, K. P., & Zhang, Y. (2014). Accepting inequality deters responsibility: how power distance decreases charitable behavior. Journal of Consumer Research, 41(2), 274–293.

    Article  Google Scholar 

  • Xie, C., Bagozzi, R. P., & Grønhaug, K. (2015). The role of moral emotions and individual differences in consumer responses to corporate green and non-green actions. Journal of the Academy of Marketing Science, 43(3), 333–356.

    Article  Google Scholar 

  • Ximénez, C., Maydeu-Olivares, A., Shi, D., & Revuelta, J. (2022). Assessing cutoff values of SEM fit indices: Advantages of the unbiased SRMR index and its cutoff criterion based on communality. Structural Equation Modeling, 29(3), 368–380.

    Article  Google Scholar 

Download references

Acknowledgements

We thank the editor and the reviewers for their constructive comments, AiMark for providing the Schwartz Value Survey data and Arvind Malhotra for providing the E-S-QUAL data. The participation of Alberto Maydeu-Olivares was supported, in part, by the Research Center for Child Well-Being (NIGMS P20GM130420), grant PID2020-119755GB-I00 funded by AEI 10.13039/501100011033, and AGAUR grant 1237SGR2017. The cross-national data for the Schwartz Value Survey as well as Mplus and R input files for both empirical applications can be found on: https://www.dropbox.com/sh/tg6573rsx27o21c/AAApQpxdvtZwWoKgosPVUVu-a?dl=0.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jan-Benedict E. M. Steenkamp.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

John Hulland served as editor for this article.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 71 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Steenkamp, JB.E.M., Maydeu-Olivares, A. Unrestricted factor analysis: A powerful alternative to confirmatory factor analysis. J. of the Acad. Mark. Sci. 51, 86–113 (2023). https://doi.org/10.1007/s11747-022-00888-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11747-022-00888-1

Keywords

Navigation