Abstract
How should we measure people’s perceptions of—and attitudes about—economic inequality? A recent literature seeks to quantify the level of inequality that people, especially Americans, perceive and prefer in society. These findings have garnered much attention from both social scientists and the public. But many of the methods used in this literature are either known to have methodological issues or have not been thoroughly compared against other methods. Thus it is not clear which, if any, are valid and reliable measures of perceived, or preferred, inequality. To assess these measures, we conducted a large web-based study (N = 831) to compare key methods for measuring perceived inequality and their related justice attitudes. In addition to comparing the resultant summary statistics, we assess how well the different measures correlate with each other and with Likert scale measures of perceived inequality. Our analysis reveals a range of issues with these measures, including failure to provide logical responses, large method effects on point estimates of inequality, and low correlations between methods and with criteria measures. We conclude our analysis with three recommendations for researchers aiming to measure inequality perceptions and preferences.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
To recruit participants, we posted a task on mTurk’s online marketplace, where ‘Workers’ (people from the general public with an mTurk worker account) can perform short tasks posted by ‘Requesters’ (including academic researchers, but also many commercial and industrial groups, e.g., companies crowdsourcing human responses for machine learning). Litman and Robinson (2021) provide full detail on Mechanical Turk. Overall, 54% of Workers who clicked on the description of our study then participated in the survey.
We will report the results for social mobility measures in a different manuscript.
We defined income inequality as “the size of the difference in income between people with the highest incomes in a society and people with the lowest incomes.” While this definition does not satisfy all principles of inequality measurement, e.g., scale invariance (Hao & Naiman, 2010), and does not specify the nature of the population the participant should calculate inequality within, e.g., only full-time workers, similar phrasing used in the GSS variable eqwlth about “income differences between rich and poor” is typically understood as referring to attitudes about income inequality in general (McCall, 2013; McCall et al., 2017). We were also concerned with defining the concept in a way that parallels definitions in related work (e.g., Chambers et al., 2014; Eriksson & Simpson, 2012; McCall, 2013; Norton & Ariely, 2011), but which would not further create unnecessary cognitive difficulty. We therefore used this definition to activate the concept of income inequality.
- $${\rm{Ideal}}\,{\rm{inequality}}\,{\rm{ratio}} = \ln \left( {\frac{{{\rm{Ideal}}\,{\rm{CEO\;}}\,{\rm{income}}}}{{{\rm{Ideal}}\,{\rm{factory}}\,{\rm{worker}}\,{\rm{income}}}}} \right).$$
References
Arsenio, W. F. (2018). The wealth of nations: International judgments regarding actual and ideal resource distributions. Current Directions in Psychological Science, 27(5), 357–362
Ashok, V., Kuziemko, I., & Washington, E. (2015). Support for redistribution in and age of rising inequality: New stylized facts and some tentative explanations. National Bureau of Economic Research Working Paper No. 21529
Auspurg, K., Hinz, T., & Sauer, C. (2017). Why should women get less? Evidence on the gender pay gap from multifactorial survey experiments. American Sociological Review, 82(1), 179–210
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012). Evaluating online labor markets for experimental research: Amazon.com’s mechanical turk. Political Analysis, 20(12), 351–368
Cech, E. A. (2017). Rugged meritocratists: the role of overt bias and the meritocratic ideology in trump supporters’ opposition to social justice efforts. Socius, 3, 1–20
Chambers, J. R., Swan, L. K., & Heesacker, M. (2014). Better off than we know: Distorted perceptions of incomes and income inequality in America. Psychological Science, 25(2), 613–618
Cheng, S., & Wen, F. (2019). Americans overestimate the intergenerational persistence in income ranks. Proceedings of the National Academy of Sciences, 116(28), 13909–13914
Clay-Warner, J., Robinson, D. T., Smith-Lovin, L., Rogers, K. B., & James, K. R. (2016). Justice standard determines emotional responses to over-reward. Social Psychology Quarterly, 79(1), 44–67
Clifford, S., Jewell, R. M., & Waggoner, P. D. (2015). Are samples drawn from mechanical Turk valid for research on political ideology? Research and Politics, 2(4), 1–9
Davidai, S., & Gilovich, T. (2018). How should we think about Americans’ beliefs about economic mobility? Judgment and Decision Making, 13, 297–308
Dawtry, R. J., Sutton, R. M., & Sibley, C. G. (2015). Why wealthier people think people are wealthier, and why it matters: From social sampling to attitudes to redistribution. Psychological Science, 26(9), 1389–1400
Eriksson, K., & Simpson, B. (2012). What do Americans know about inequality? It depends on how you ask them. Judgment and Decision Making, 7(6), 741–745
Eriksson, K., & Simpson, B. (2013). The available evidence suggests the percent measure should not be used to study inequality: Reply to Norton and Ariely. Judgment and Decision Making, 8(3), 395–396
Evans, M. D. R., Evans, J., & Kolosi, T. (1992). Images of class: Public perceptions in Hungary and Australia. American Sociological Review, 57(4), 461–482
Gimpelson, V., & Treisman, D. (2018). Misperceiving inequality. Economics and Politics, 30(1), 27–54
Hao, L., Naiman, D. Q. (2010). Assessing inequality. Sage Publications.
Hauser, O. P., & Norton, M. I. (2017). (Mis)Perceptions of inequality. Current Opinion in Psychology, 18, 21–25
Hegtvedt, K. A., & Isom, D. (2014). Inequality: A matter of justice?” In J. D. MacLeod, E. J. Lawler, & M. Schwalbe (Eds.), Handbook of the social psychology of inequality (pp. 65–94). Springer.
Heiserman, N., & Simpson, B. (2017). Higher inequality increases the gap in the perceived merit of the rich and poor. Social Psychology Quarterly, 80(3), 243–253
Heiserman, N., Simpson, B., & Willer, R. (2020). Judgments of economic fairness are based more on perceived economic mobility than perceived inequality. Socius, 6, 1–12
Hertel, F. R., & Schöneck, N. M. (2019). Conflict perceptions across 27 OECD countries: The roles of socioeconomic inequality and collective stratification beliefs. Acta Sociologica 1–19. doi: https://doi.org/10.1177/0001699319847515
Homans, G. C. (1974). Social behavior: Its elementary forms. Harcourt Brace.
Hunt, M. O., & Bullock, H. E. (2016). Ideologies and beliefs about poverty. In D. Brady, & L. M. Burton (Eds.), The Oxford handbook of the social science of poverty (pp. 93–116). New York: Oxford University Press.
Jasso, G., & Wegener, B. (1997). Methods for empirical justice analysis: Part 1. Framework, models, and quantities. Social Justice Research, 10(4), 393–430
Kelley, J., & Evans, M. D. R. (1993). The legitimation of inequality: Occupational earnings in nine nations. American Journal of Sociology, 99(1), 75–125
Kiatpongsan, S., & Norton, M. I. (2014). How much (more) should CEOs make? A universal desire for more equal pay. Perspectives on Psychological Science, 9(6), 587–593
Klein, E. (2013). This viral video is right: We need to worry about wealth inequality. The Washington Post. Retrieved from (https://www.washingtonpost.com/news/wonk/wp/2013/03/06/this-viral-video-is-right-we-need-to-worry-about-wealth-inequality/)
Kluegal, J. R., & Smith, E. R. (1986). Beliefs about inequality: Americans' views of what is and what ought to be. Transaction
Kuhn, A. (2019). The individual (mis-)perception of wage inequality: Measurement, correlates and implications. Empirical Economics. doi: https://doi.org/10.1007/s00181-019-01722-4
Litman, L., & Robinson, J. (2021). Conducting online research on Amazon Mechanical Turk and beyond. Sage Publications
Litman, L., Robinson, J., & Abbercock, T. (2017). TurkPrime.com: A versatile crowdsourcing data acquisition platform for the behavioral sciences. Behavior Research Methods, 49(2), 433–442
Markovsky, B., & Eriksson, K. (2012). Comparing direct and indirect measures of just rewards. Sociological Methods and Research, 41(1), 199–216
McCall, L., Burk, D., Laperrière, M., & Richeson, J. A. (2017). Exposure to rising inequality shapes Americans’ opportunity beliefs and policy support. Proceedings of the National Academy of Science, 114(36), 9593–9598
McCall, L. (2013). The undeserving rich: American beliefs about inequality, opportunity, and redistribution. Cambridge University Press.
Melamed, D., Liu, Y., Park, H., & Zhong, J. (2018). Referent networks predict just rewards. Sociological Focus, 51(4), 304–317
Meltzer, A. H., & Scott, R. F. (1981). A rational theory of government. Journal of Political Economy, 89(5), 914–927
Niehues, J. (2014). Subjective perceptions of inequality and redistributive preferences: An international comparison. IW-Trends Discussion Papers No. 2. Cologne Institute for Economic Research
Norton, M. I., & Ariely, D. (2011). Building a better America-one wealth quintile at a time. Perspectives on Psychological Science, 6(1), 9–12
Osberg, L., & Smeeding, T. (2006). “Fair” inequality? attitudes toward pay differentials: The United States in comparative perspective. American Sociological Review, 71(3), 450–473
Pedersen, R., & Mutz, D. C. (2018). Attitudes toward economic inequality: the illusory agreement. Political Science Research and Methods, 12, 1–17
Reynolds, J., & Xian, He. (2014). Perceptions of meritocracy in the land of opportunity. Research in Stratification and Mobility, 36, 121–137
Sauer, C., & Valet, P. (2013). Less is sometimes more: Consequences of overpayment on job satisfaction and absenteeism. Social Justice Research, 26(2), 132–150
Schneider, S. M., & Castillo, J. C. (2015). Poverty attributions and the perceived justice of income inequality: A comparison of East and West Germany. Social Psychology Quarterly, 78(3), 263–282
Schneider, S. M., & Valet, P. (2017). Relative standards and distributive justice: How social comparison orientations moderate the link between relative earnings and justice perceptions. Social Psychology Quarterly, 80(3), 276–287
Schröder, M. (2017). Is income inequality related to tolerance for inequality? Social Justice Research, 30(1), 23–47
Small, M. L. (2011). How to conduct a mixed methods study: Recent trends in a rapidly growing literature. Annual Review of Sociology, 37, 57–86
Thye, S. R. (2000). Reliability in experimental sociology. Social Forces, 78(4), 1277–1309
Trump, K.-S. (2017). Income inequality influences perceptions of legitimate income differences. British Journal of Political Science, 48(4), 929–952
Acknowledgements
This research was supported by an ASPIRE grant from the University of South Carolina and a grant to the second author from the Army Research Office (Proposal No. 74780-LS).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Heiserman, N., Simpson, B. Measuring Perceptions of Economic Inequality and Justice: An Empirical Assessment. Soc Just Res 34, 119–145 (2021). https://doi.org/10.1007/s11211-021-00368-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11211-021-00368-x