Skip to main content
SpringerLink
Log in

Evaluating the effectiveness of a police department’s early intervention system

  • Published:
Journal of Experimental Criminology Aims and scope Submit manuscript

Abstract

Objectives

Police departments around the country are implementing Early Intervention Systems (EIS) to identify officers who may be exhibiting problematic or unprofessional behaviors. The goal of EIS is to minimize officer misconduct and increase officer accountability. To evaluate whether EIS can actually differentiate “problem” from “non-problem” officers, we analyzed the performance of officers from incident reports of police–citizen interactions.

Methods

Using a blind scoring method, we evaluated performance from 1000 police reports; 500 randomly selected reports from EIS-flagged officers (treatment group) and 500 randomly selected reports from non-flagged officers (control group). Six hundred and sixty-seven reports contained relevant performance data. The interval-level metrics used to score officer performance were developed by Vila and colleagues (2016, 2018) to assess performance—expressed as a percentage—across a range of police–citizen encounters.

Results

The overall performance score assigned to officers across all 667 incident reports was 80.46% (SD = 8.75%). When separated into EIS-flagged and non-EIS–flagged incidents, performance scores were 80.63% (SD = 8.58%) compared to 80.27% (SD = 8.95). There was not a statistically significant difference between EIS-flagged and non-EIS–flagged performance.

Conclusions

The EIS evaluated does not appear to be differentiating between problem behavior and non-problem behavior. This suggests that the “thresholds” used to identify problem officers are not working effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Alpert, G. P., & Walker, S. (2000). Police accountability and early warning systems: developing policies and programs. Justice Research and Policy, 2(2), 59–72. https://doi.org/10.3818/JRP.2.2.2000.59.

    Article  Google Scholar 

  • Ariel, B., Farrar, W., & Sutherland, A. (2015). The effect of police body-worn cameras on use of force and citizens’ complaints against the police: a randomized controlled trial. Journal of Quantitative Criminology, 31(3), 509–535.

    Article  Google Scholar 

  • Ariel, B., Sutherland, A., Henstock, D., Young, J., Drover, P., Sykes, J., et al. (2017). "Contagious accountability": a global multisite randomized controlled trial on the effect of police body-worn cameras on citizens' complaints against the police. Criminal Justice And Behavior, 44(2), 293–316.

    Article  Google Scholar 

  • Bazley, T. D., et al. (2009). Early intervention program criteria: evaluating officer use of force. Justice Quarterly, 26(1), 107–124. https://doi.org/10.1080/07418820801989742.

    Article  Google Scholar 

  • Brandl, S. G., et al. (2001). Who are the complaint-prone officers?: An examination of the relationship between police officers' attributes, arrest activity, assignment, and citizens' complaints about excessive force. Journal of Criminal Justice, 29(6), 521–529. https://doi.org/10.1016/S0047-2352(01)00114-3.

    Article  Google Scholar 

  • Harris, C. J. (2009). Exploring the relationship between experience and problem behaviors. Police Quarterly, 12(2), 192–213. https://doi.org/10.1177/1098611108327314.

    Article  Google Scholar 

  • Harris, C. J. (2016) “Towards a career view of police misconduct.” Aggression and Violent Behavior, vol. 31, 2016, pp. 219–228., doi:https://doi.org/10.1016/j.avb.2016.10.001.

  • Hedberg, E., Katz, C., & Choate, D. (2017). Body-worn cameras and citizen interactions with police officers: estimating plausible effects given varying compliance levels. Justice Quarterly, 34(4), 627–651.

    Article  Google Scholar 

  • Helsby, J., Stageman, D. L., et al. (2018). Early intervention systems: predicting adverse interactions between police and the public. Criminal Justice Policy Review, 29(2), 190–209. https://doi.org/10.1177/0887403417695380.

    Article  Google Scholar 

  • Independent Commission on the Los Angeles Police Department (1991), Report of the Independent Commission on the Los Angeles Police Department, Los Angeles, CA.

  • Jackson, B., et al. (2017). Managing officer behavioural risk using early intervention systems: addressing system design challenges for law enforcement and corrections environments. Policing : A Journal of Policy and Practice, 11(1), 103. https://doi.org/10.1093/police/paw028.

    Article  Google Scholar 

  • James, L., & James, S. (2017). Crisis intervention team (CIT) metrics: a novel method of measuring police performance during encounters with people in crisis. Mental Health and Addiction Research, 2(2), 1–4.

    Google Scholar 

  • Kane, M., & Trochim, W. M. K. (2007). Concept mapping for planning and evaluation. Thousand Oaks, CA: Sage.

    Book  Google Scholar 

  • Kappeler, V. E., Sluder, R. D., & Alpert, G. P. (1998). Forces of deviance: understanding the dark side of policing. Prospect Heights, IL: Waveland Press.

    Google Scholar 

  • Lersch, K. M. (2002). Are citizen complaints just another measure of officer productivity? An analysis of citizen complaints and officer activity measures. Police Practice and Research, 3(2), 135–147. https://doi.org/10.1080/15614260290033639.

    Article  Google Scholar 

  • Lersch, M., Kim, et al. (2006). Early intervention programs: an effective police accountability tool, or punishment of the productive? Policing: An International Journal of Police Strategies & Management, 29(1), 58–76. https://doi.org/10.1108/13639510610648485.

    Article  Google Scholar 

  • Mitchell, R., Ariel, B., Firpo, M., Fraiman, R., Del Castillo, F., Hyatt, J., et al. (2018). Measuring the effect of body-worn cameras on complaints in Latin America: The case of traffic police in Uruguay. Policing-An International Journal Of Police Strategies & Management, 41(4), 510–524.

    Article  Google Scholar 

  • Newman, J. (2015). Program that flags Chicago cops at risk of misconduct misses most officers. The Chicago Reporter.

  • Shjarback, J. A. (2015). Emerging early intervention systems: an agency-specific pre-post comparison of formal citizen complaints of use of force. Policing: A Journal of Policy and Practice, 9(4), 314–325. https://doi.org/10.1093/police/pav006.

    Article  Google Scholar 

  • Shjarback, J. A., Pyrooz, D. C., Wolfe, S. E., & Decker, S. H. (2017). De-policing and crime in the wake of Ferguson: racialized changes in the quantity and quality of policing among Missouri police departments. Journal of Criminal Justice, 50, 42–52.

    Article  Google Scholar 

  • Thurstone, L. L. (1959). Measurement of values. Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Thurstone, L. L., & Chave, E. J. (1929). The measurement of attitude: a psychophysical method and some experiments with a scale for measuring attitude toward the church. Chicago, IL: University of Chicago Press.

    Book  Google Scholar 

  • U.S. Commission on Civil Rights. (1981). Who is guarding the guardians? Washington, DC: Government Printing Office.

    Google Scholar 

  • Vila, B., James, L., James, S. M., & Waggoner, L. B. (2012). “Final report: developing a common metric for evaluating police performance in deadly force situations”, Grant No. 2008IJCX0015. Washington, DC: National Institute of Justice.

    Book  Google Scholar 

  • Vila, B., James, L., & James, S. M. (2014). “Final report: empowering the strategic corporal: training young warfighters to be socially adept with strangers in any culture”, Grant No. W911NF-12-0039. Arlington, VA: Defense Advanced Research Projects Agency.

    Google Scholar 

  • Vila, B., James, S. M., & James, L. (2016). Novel process for developing metrics that measure what police do. In Interservice/Industry Training, Simulation and Education Conference (I/ITSEC) proceedings, Orlando, Florida.

  • Vila, B., James, S., & James, L. (2018). How police officers perform in encounters with the public. Policing: An International Journal, 41(2), 215–232 https://doi.org/10.1108/PIJPSM-11-2016-0166.

  • Walker, S. (2010). Introduction to early intervention systems. In R. G. Dunham & G. P. Alpert (Eds.), Critical issues in police: Contemporary readings (6th ed., pp. 181–196). Long Grove, IL: Waveland Press, Inc..

    Google Scholar 

  • Walker, S., & Alpert, G. P. (2002). Early warning systems as risk management for police. In K. M. Lersch (Ed.), Policing and Misconduct (pp. 219–230). Upper Saddle River, NJ: Prentice-Hall.

    Google Scholar 

  • Walker, S., Alpert, G., & Kenney, D. (2000). Responding to the problem police officer: A national evaluation of early warning systems. Final report to the National Institute of Justice.

  • White, M. D., Gaub, J. E., & Todak, N. (2017). Exploring the potential for body-worn cameras to reduce violence in police–citizen encounters. Policing: A Journal of Policy and Practice, 12(1), 66–76.

    Google Scholar 

  • Worden, R. E., Kim, M., Harris, C. J., Pratte, M. A., Dorn, S. E., & Hyland, S. S. (2013). Intervention with problem officers: an outcome evaluation of an EIS intervention. Criminal Justice and Behavior, 40(4), 409–437.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Elson S. Floyd College of Medicine, Washington State University, Spokane, WA, USA

    Stephen James

  2. College of Nursing, Washington State University, Spokane, WA, USA

    Lois James

  3. Department of Criminal Justice and Criminology, Washington State University, Pullman, WA, USA

    Liz Dotson

Authors
  1. Stephen James
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lois James
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liz Dotson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lois James.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

James, S., James, L. & Dotson, L. Evaluating the effectiveness of a police department’s early intervention system. J Exp Criminol 17, 457–471 (2021). https://doi.org/10.1007/s11292-019-09397-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11292-019-09397-8

Keywords

Search

Navigation