Skip to main content
SpringerLink
Log in

Measuring Racial Disparities in Police Use of Force: Methods Matter

  • Original Paper
  • Published:
Journal of Quantitative Criminology Aims and scope Submit manuscript

Abstract

Objectives

To understand the impact of measurement and analytic choices on assessments of police use of force (UOF) and racial disparities therein.

Methods

We collected and standardized UOF data (N = 9982 incidents) from a diverse set of 11 police departments, and measured departments’ aggregate force severity in five ways. We assessed the sensitivity of racial disparities in UOF severity to a series of analytic choices, using a 5 × 2 × 2 × 2 design comparing force severity to population and arrest benchmarks, using two definitions of minority group (Black/Nonwhite), and two modes of comparison (ratios/differences).

Results

Significant racial disparities were observed under most analytic choices in most departments. However, lethal force was rare, and estimates of lethal force disparities were statistically uncertain, as were departments’ relative ranks as equitable or disparate. Ratios of minority to White force severity were less sensitive to measurement differences within measures including nonlethal force. The choice of a population or arrest benchmark had implications for which departments emerged as highly disparate, while focal minority group and mode of comparison had less systematic effects.

Conclusions

Given increased scrutiny of police activity by advocates and policymakers, it is important to understand how measurement and other analytic choices affect our understanding of equity in police practices. Our findings demonstrate that analytical decisions interact in complex ways and that standardization is essential when comparing multiple departments. We recommend comprehensive data collection that includes nonlethal as well as lethal force, and make recommendations for measuring and contextualizing racial disparities in UOF and other police activity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. To expand upon dichotomous UOF concepts and operationalization, Alpert and Dunham (1997) constructed a measure referred to as the “force factor”, that relates information on officer force level to documented citizen resistance. Combining identically scaled, ordinal officer force level and citizen resistance level, one computes the force factor by subtracting citizen resistance from officer force (i.e., officer force—citizen resistance = force factor). Force factors with positive values then indicate officer force exceeding citizen resistance, while negative values indicate officer force less than citizen resistance, and values of zero indicate commensurate officer force and citizen resistance. Terrill et al. further develop the force factor by computing a force factor for each “sequence” or dyad of officer-citizen behaviors within an incident, which can then be summarized into an incident-level measure (Terrill et al. 2003). The force factor is envisioned as a tool to support police administrators in their use and interpretation of UOF information (ibid.). While the force factor has clear intradepartmental applications in the comparison of officers, work units, districts, or other officer groupings, variation in data collection practices across departments is a barrier to its use in interdepartmental comparisons. Limited or structurally different detail around citizen resistance, as well as variation in reliability that seems particularly salient in regards to police documentation of citizen resistance, precludes us from applying the force factor in the current work.

  2. The Oakland force ranking system is detailed in Appendix A in supplementary material.

  3. Department I, which reported the most lethal force per year, did not indicate subject race on any incidents of lethal force, and was therefore not considered in any disparity measures or rankings in CS2.

  4. As an additional sensitivity analysis, we also examined arrests for violent crime as an additional benchmark, with results presented in appendices. Police UOF often stems from events unrelated to violent crime (Zimring 2017); however, officers may form their beliefs about suspicion based on perceived propensities for violence (even if these perceptions are driven by stereotypes), making violent crime arrests a benchmark worth considering. The above-mentioned concern that arrest benchmarks may reflect racially disparate policing and arrest practices applies to this more specific benchmark as well; however, the extent to which this distortion disproportionately affects disparity estimates benchmarked to offense-specific arrests (rather than total arrests) is theoretically ambiguous.

  5. Observations were sampled with replacement from within each department, across all force categories. Different samples were drawn for Black/White and Nonwhite/White analyses. Replicating the analysis using 1000 bootstrapped samples produced substantively similar results.

  6. Of the 32 combinations of coding scheme, benchmark, minority group, and difference measure that involved lethal force, 7 had rank-order correlations between .5 and .75, 5 of which were equal to.6 or below, and statistically nonsignificant. All rank-order correlations below .5 were statistically nonsignificant.

  7. Disparity ratios benchmarked to violent crime arrests were attenuated further. Fewer departments recorded significant differences where Black subjects experienced significantly more severe force, some indicated that Black and White subjects experiences statistically similar benchmarked force severity, and some departments recorded disparities in which Black subjects experienced significantly less severe force than White subjects. Disparity rankings also differed when using a violent arrest benchmark, with departments previously among the most equitable (e.g., Department J, G, and C) moving to the middle of the distribution, and departments previously in the middle ranking toward the more equitable (e.g., department B). Details are presented in Appendix E in supplementary material.

  8. Notably, despite the changes in rankings observed when using a violent arrest rather than total arrest benchmark (presented in Appendix E in supplementary material), Department F remains the most disparate in lethal force, regardless of benchmark used.

  9. Department A also indicated that differences in UOF against Nonwhite and White subjects were statistically insignificant across schemes and benchmark choices, and Department K indicated insignificance in CS3 with a population benchmark. Departments A, and J had arrest-benchmarked differences that were statistically insignificant across all schemes, while Department G’s arrest-benchmarked differences were insignificant only in CS3 and CS5a.

  10. Disparities were attenuated in magnitude when benchmarked to arrests for violent crime, and like Black-White disparities, some departments’ Nonwhite-White ratios also indicated significantly more severe force against Whites than Nonwhites, some indicated statistically similar force severity, and others’ continued to indicate more severe force against Nonwhites. Departmental rankings also varied considerably, though the departments ranked as more disparate tended to be more robust than those ranked as more equitable.

  11. Unlike ratio measures of disparity, difference measures were larger in magnitude when benchmarked to violent arrests than total events, due to scale differences in the benchmarks. Like ratio measures, the rankings also varied when benchmarking to violent rather than total arrests.

  12. These are provided in Appendices E and F.

References

  • Alpert GP, Dunham RG (1997) The force factor: measuring police use of force relative to suspect resistance. Police Executive Research Forum, Washington, DC

    Google Scholar 

  • American Civil Liberties Union of Illinois (2015) Stop and Frisk in Chicago. Retrieved from Chicago, IL

  • Ayres I, Borowsky J (2008) A study of racially disparate outcomes in the los Angeles police department. Retrieved from American Civil Liberties Union

  • Bureau of Justice Statistics (2017) Arrest-related deaths. Retrieved from https://www.bjs.gov/index.cfm?ty=tp&tid=82

  • Centers for Disease Control and Prevention (2017) National Vital Statistics Program: Fatal Injury Data. Retrieved July 4, 2017, from U.S. Centers for Disease Control and Prevention https://www.cdc.gov/injury/wisqars/fatal.html

  • Close BR, Mason PL (2007) Officer characteristics and racially biased policing. Rev Law Econ 3(2):263–321

    Article  Google Scholar 

  • Davis E, Whyde A, Langton L (2018) Contacts between Police and the Public, 2015. Retrieved from Washington, DC: https://www.bjs.gov/content/pub/pdf/cpp15.pdf on January 27, 2020

  • Eberhardt JL (2016) Strategies for change: Research initiatives and recommendations to improve police-community relations in Oakland, Calif. Stanford University, SPARQ (Social Psychological Answers to Real-world Questions)

  • Engel RS, Calnon JM (2004) Comparing benchmark methodologies for police-citizen contacts: traffic stop collection for the Pennsylvania State Police. Police Quart 7(1):97–125. https://doi.org/10.1177/1098611103257686

    Article  Google Scholar 

  • Fagan J (2010) Expert testimony. Floyd et al. v. City of New York et al. 08 Civ 1034 (SAS)

  • Fagan J, Geller A, Davies G, West V (2010) Street stops and Broken Windows revisited: the demography and logic of proactive policing in a safe and changing city. In: Rice SK, White MD (eds) Race, ethnicity, and policing: New and essential readings. NYU Press, New York

    Google Scholar 

  • Federal Bureau of Investigations Uniform Crime Reporting Program (2017) National Use-of-Force Data Collection. Retrieved from https://ucr.fbi.gov/use-of-force

  • Feldman J (2016) Roland Fryer is wrong: there is racial bias in shootings by police. Retrieved from http://scholar.harvard.edu/jfeldman/blog/roland-fryer-wrong-there-racial-bias-shootings-police

  • Fryer RG Jr (2019) An empirical analysis of racial differences in police use of force. J Polit Econ 127(3):1210–1261

    Article  Google Scholar 

  • Geller A (2020) Youth-police contact: burdens and inequities in an adverse childhood experience. Unpublished manuscript (under review)

  • Geller A, Fagan J, Tyler TR, Link B (2014) Aggressive policing and the mental health of young urban men. Am J Public Health 104(12):2321–2327

    Article  Google Scholar 

  • Gelman A, Fagan J, Kiss A (2007) An analysis of the NYPD’s stop-and-frisk policy in the context of claims of racial bias. J Am Stat Assoc 102(479):813–822

    Article  Google Scholar 

  • Goff PA, Lloyd T, Geller A, Raphael S, Glaser J (2016a) The science of justice: race, arrests, and police use of force. Center for Policing Equity, Los Angeles

    Google Scholar 

  • Goff PA, Obermark D, LaVigne N, Yahner J, Geller A (2016b) The science of policing equity: measuring fairness in the Austin police department. Center for Policing Equity, New York

    Google Scholar 

  • Grogger J, Ridgeway G (2006) Testing for racial profiling in traffic stops from behind a veil of darkness. J Am Stat Assoc 101(475):878–887. https://doi.org/10.1198/016214506000000168

    Article  Google Scholar 

  • Klinger DA (2010) Can Police training affect the use of force on the streets? The metro-dade violence reduction field experiment. In: McCoy C (ed) Holding police accountable. The Urban Institute Press, Washington, DC

    Google Scholar 

  • Knowles J, Persico N, Todd P (2001) Racial bias in motor vehicle searches: theory and evidence. J Polit Econ 109(1):203–229

    Article  Google Scholar 

  • Knox D, Will L, Jonathan M (2019) Administrative records mask racially biased policing (December 21, 2019). Available at SSRN: https://ssrn.com/abstract=3336338; http://dx.doi.org/10.2139/ssrn.3336338

  • Legewie J (2016) Racial profiling and use of force in police stops: how local events trigger periods of increased discrimination. Am J Sociol 122(2):379–424

    Article  Google Scholar 

  • Lovrich NP, Gaffney MJ, Mosher CC, Pratt TC, Pickerill MJ (2007) Results of the monitoring of WSP traffic stops for biased policing: analysis of WSP Stop, Citation, Search and Use of Force Data and Results of the Use of Observational Studies for Denominator Assessment. Retrieved from Seattle, WA: http://www.wsp.wa.gov/publications/reports/wsu_2007_report.pdf

  • MacDonald J, Manz PW, Alpert GP, Dunham RG (2003) Police use of force: examining the relationship between calls for service and the balance of police force and suspect resistance. J Crim Justice 31:119–127

    Article  Google Scholar 

  • Manski CF, Nagin D (2017) Assessing benefits, costs, and disparate racial impacts of confrontational proactive policing. Proc Natl Acad Sci USA 114(35):9308–9313. https://doi.org/10.1073/pnas.1707215114

    Article  Google Scholar 

  • McElvain JP, Kposowa AJ (2008) Police officer characteristics and internal affairs investigations for use of force allegations. J Crim Justice 35(4):505–521. https://doi.org/10.1177/0093854807313995

    Article  Google Scholar 

  • Missouri House of Representatives (2019) 100th General Assembly, House Bill 484: An Act to repeal section 590.650, RSMo, and to enact in lieu thereof one new section relating to prohibitions against discriminatory policing. Downloaded from https://www.house.mo.gov/billtracking/bills191/hlrbillspdf/0897H.02I.pdf on February 3, 2020

  • Nix J, Campbell BA, Byers EH, Alpert GP (2017) A bird’s eye view of civilians killed by police in 2015. Criminol Public Policy 16(1):309–340. https://doi.org/10.1111/1745-9133.12269

    Article  Google Scholar 

  • Oakland Police Department (2018) Use of force in policing. Retrieved from http://www2.oaklandnet.com/government/o/OPD/a/data/useofforce/index.htm on July 5, 2018

  • Pierson E, Simoiu C, Overgoor J, Corbett-Davies S, Jenson D, Shoemaker A, Ramachandran V, Barghouty P, Phillips C, Shroff R, Goel S (2019) A large-scale analysis of racial disparities in police stops across the United States. Palo Alto, CA: Stanford Computational Policy Lab. Downloaded from https://5harad.com/papers/100M-stops.pdf. April 25, 2019

  • Police Data Initiative (2017) The Police Data Initiative. Retrieved from https://www.policedatainitiative.org/about/. June 10, 2019

  • Prenzler T, Porter L, Alpert GP (2013) Reducing police use of force: case studies and prospects. Aggress Violent Behav 18:343–356

    Article  Google Scholar 

  • Ridgeway G (2007) Analysis of racial disparities in the New York Police Department’s Stop, Question, and Frisk Practices. Retrieved from Santa Monica, CA

  • Ridgeway G, MacDonald JM (2009) Doubly robust internal benchmarking and false discovery rates for detecting racial bias in police stops. J Am Stat Assoc 104(486):661–668

    Article  Google Scholar 

  • Schuck AM (2004) The masking of racial and gender disparity in police use of force: the effects of gender and custody status. J Crim Justice 21:557–564

    Article  Google Scholar 

  • Sethi R (2016) Police use of force: notes on a study. Retrieved from http://rajivsethi.blogspot.com/2016/07/police-use-of-force-notes-on-study.html

  • Sewell AA, Jefferson KA, Lee H (2016) Living under surveillance: gender, psychological distress, and stop- question-and-frisk policing in New York City. Soc Sci Med 159(3):1–13. https://doi.org/10.1016/j.socscimed.2016.04.024

    Article  Google Scholar 

  • Shedd C (2015) Unequal City: Race, schools, and perceptions of injustice. Russell Sage Foundation, New York

    Google Scholar 

  • Sinyangwe S (2019) Evaluating Policing in San Diego. Downloaded from https://policescorecard.org/sandiego, February 5, 2020

  • Skogan WG, Frydl K (eds) (2004) Fairness and effectiveness in policing: the evidence. National Academies Press, Washington, DC

    Google Scholar 

  • Smith MR, Kaminiski RJ, Alpert GP, Fridell LA, MacDonald J, Kubu B (2010) A multi-method evaluation of police use of force outcomes: final report to the National Institute of Justice. Retrieved from Washington, DC: https://www.ncjrs.gov/pdffiles1/nij/grants/231176.pdf

  • Stone C, Travis J (2011) Toward a new professionalism in policing. National Institute of Justice, Washington, DC

    Google Scholar 

  • Swaine J, Laughland O, Lartey J, McCarthy C (2017) The counted: people killed by police in the United States. Retrieved from https://www.theguardian.com/us-news/ng-interactive/2015/jun/01/the-counted-police-killings-us-database. June 10, 2018

  • Taylor B, Alpert GP, Kubu B, Woods D, Dunham RG (2011) Changes in officer use of force over time: a descriptive analysis of a national survey. Policing An Int J Police Strat Manage 34(2):211–232

    Article  Google Scholar 

  • Terrill W, Mastrofski SD (2002) Situational and officer-based determinants of police coercion. Justice Q 19(2):215–248. https://doi.org/10.1080/07418820200095221

    Article  Google Scholar 

  • Terrill W, Alpert GP, Dunham RG, Smith MR (2003) A management tool for evaluating police use of force: an application of the force factor. Police Quart 6(2):150–171

    Article  Google Scholar 

  • The Washington Post (2017) Fatal Force. Retrieved July 4, 2017 https://www.washingtonpost.com/graphics/national/police-shootings-2017/

  • Thorkildsen Z, Wohl E, Walther B, Bryson B, Cunningham B (2019) Maricopa county sheriff’s office traffic stops analysis report: July 2017–December 2018. Maricopa County Sheriff’s Office, Phoenix

    Google Scholar 

  • Tregle B, Nix J, Alpert GP (2019) Disparity does not mean bias: making sense of observed racial disparities in fatal officer-involved shootings with multiple benchmarks. J Crim Justice 42(1):18–31. https://doi.org/10.1080/0735648X.2018.1547269

    Article  Google Scholar 

  • United States (1968) Report of the National Advisory Commission on Civil Disorders. Washington, DC: Kerner Commission: US GPO

  • United States Department of Justice (2017) The civil rights division’s pattern and practice police reform work: 1994-Present. Retrieved from https://www.justice.gov/crt/file/922421/download. January 27, 2020

  • Weisburd D, Majmandur M (eds) (2018) Proactive policing: effects on crime and communities. National Academies of Science, Engineering, and Medicine, Washington, DC

    Google Scholar 

  • Weitzer R (2017) Theorizing racial discord over policing before and after Ferguson. Justice Q 34(7):1129–1153. https://doi.org/10.1080/07418825.2017.1362461

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Criminology, Law, and Society, University of California, Irvine, 2315 Social Ecology 2, Irvine, CA, 92697, USA

    Amanda Geller

  2. Center for Policing Equity, Yale University, New Haven, CT, USA

    Phillip Atiba Goff & Tracey Lloyd

  3. Heinz College, Carnegie Mellon University, Pittsburgh, PA, USA

    Amelia Haviland

  4. California Policy Lab, University of California, Los Angeles, Los Angeles, CA, 90095, USA

    Dean Obermark

  5. Goldman School of Public Policy, University of California, Berkeley, Berkeley, CA, USA

    Jack Glaser

Authors
  1. Amanda Geller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Phillip Atiba Goff
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tracey Lloyd
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Amelia Haviland
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dean Obermark
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jack Glaser
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amanda Geller.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 522 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Geller, A., Goff, P.A., Lloyd, T. et al. Measuring Racial Disparities in Police Use of Force: Methods Matter. J Quant Criminol 37, 1083–1113 (2021). https://doi.org/10.1007/s10940-020-09471-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10940-020-09471-9

Keywords

Search

Navigation