Abstract
Whereas using artificial intelligence (AI) to predict natural hazards is promising, applying a predictive policing algorithm (PPA) to predict human threats to others continues to be debated. Whereas PPAs were reported to be initially successful in Germany and Japan, the killing of Black Americans by police in the US has sparked a call to dismantle AI in law enforcement. However, although PPAs may statistically associate suspects with economically disadvantaged classes and ethnic minorities, the targeted groups they aim to protect are often vulnerable populations as well (e.g., victims of human trafficking, kidnapping, domestic violence, or drug abuse). Thus, determining how to enhance the benefits of PPA while reducing bias through better management is important. In this paper, we propose a policy schema to address this issue. First, after clarifying relevant concepts, we examine major criticisms of PPAs and argue that some of them should be addressed. If banning AI or making it taboo is an unrealistic solution, we must learn from our errors to improve AI. We next identify additional challenges of PPAs and offer recommendations from a policy viewpoint. We conclude that the employment of PPAs should be merged into broader governance of the social safety net and audited publicly by parliament and civic society so that the unjust social structure that breeds bias can be revised.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
For example, the RAND Safety and Justice Program defines predictive policing as “the application of analytical techniques—particularly quantitative techniques—to identify likely targets for police intervention and crime prevention or solve past crimes by making statistical predictions” (Perry et al., 2013, pp. 1–2). It also notes that “[t]he use of statistical and geospatial analyses to forecast crime levels has been around for decades.” “[A] surge of interest in analytical tools that draw on very large data sets to make predictions in support of crime prevention” occurs only recently (Perry et al., 2013, p. 2).
For example, the applications include PredPol, PreCobs, Hunchlab and Crime Anticipation System, etc. (Hardyns & Rummens, 2018).
See also Kleinberg et al. (2018) and Skeem and Lowenkamp (2016). If the inclusion of feature-specific factors helps advance justice, it may be permissible to do so. For example, recently, in Wisconsin v. Loomis (2016), the Wisconsin Supreme Court noted that women are typically less likely to participate in crime and held that a trial court’s use of an algorithmic risk assessment that took gender into account served the nondiscriminatory purpose of promoting accuracy.
As one of our anonymous reviewers pointed out, the problem of ethics washing is usually the failure to track if the ethics guidelines are actually implemented in practice. AI ethics guidelines have told us “the ‘what’ of AI ethics,” but still, there is “the ‘how’ [question] of AI ethics”—how to translate these guidelines into practices (Morley et al., 2019). Ethics washing is a pointer to the need to bridge the gap between the ethics discourses on the one hand and the technical ones on the other.
In Japan, for example, most factors affecting public cooperation with the police are not racial (Tsushima & Hamai, 2015). Also, in Germany and Japan, albeit they still have room to improve, area-based predictive policing is reported to be initially successful (Egbert & Krasmann, 2020; Ohyama & Amemiya, 2018).
As one of our anonymous reviewers pointed out, a lack of proof of the lack of usefulness of PPAs is not required to stop supporting it. The burden of argument should be on those who want to support PPAs. It is a fair point. We argue that in order to measure the effectiveness of PPAs, PPAs must be conceived in the broader context of law enforcement operations. From this perspective, the usefulness of PPAs is to be integrated into broader systems. For example, in Chicago, predictive algorithms are part of the police department’s strategic decision support centers (SDSCs). We can take the following passage from a recent report of SDSCs as an upbeat assessment of the deployment of PPAs (Hollywood et al., 2019, p. 70):
As a result, policing decisions can be made with a much higher level of quality—timelier, more complete, and more accurate—than was typically possible before.... More broadly, we see SDSCs as a promising model for improving law enforcement agencies’ awareness of their communities, improving their decisionmaking, and carrying out more effective and more efficient operations that lead to crime reductions and other policing benefits.
Besides, police departments adopting the technologies must acknowledge these tools’ vulnerabilities and the following limitations of the conclusions drawn to make room for auditing and improving these technologies' performance. For example, we may establish auditing mechanisms to check the quality of inputs from the algorithms. While predictive policing programs are not completely bias-free, it is not a sufficient reason to dismantle PPAs.
There are other challenges, such as privacy, that we have not discussed in this article but have elsewhere. For example, a dilemma is that if the algorithmic prediction is accurate, it must be trained on a massive amount of biometric data that risk privacy, but if it is not accurate, the false positives threaten the human rights of misidentified targets. Please see Hung (2020), Hung and Yen (2020), and Lin et al. (2020).
Here is an example. The Crime and Victimization Risk Model (CVRM) was a statistical model used by the Chicago Police Department. It used arrest and crime incident data from within the Chicago Police Department’s record management systems to estimate an individual’s risk for becoming a party to violence. As shown on the department’s Violence Reduction Strategy web page (https://home.chicagopolice.org/information/violence-reduction-strategy-vrs/), the CVRM was “for the sole purpose of finding just the small group that may be at highest risk, so that the details of their crime records can be studied by experts for purposes of prioritizing the Custom Notifications program.” We do not need to know the complete details of how the algorithm works as long as we can decide whether it fulfilled its assigned purpose. It was reported that “among the individuals with the highest CVRM risk scores, approximately 1 in 3 will be involved in a shooting or homicide in the next 18 months” (Illinois Institute of Technology, 2019, p. 3). As reported on the department’s Violence Reduction Strategy web page (https://home.chicagopolice.org/information/violence-reduction-strategy-vrs/), that was a reasonably effective piece of information “to help to prioritize the Custom Notifications process,” given that “a Chicago resident with no arrests in the past four years has about a 1 in 2300 chance of being a shooting victim [in the next 18 months].”
Take CVRM as an example again. In a 2019 review, the RAND Corporation found that the Chicago Police Department initially was not fully transparent about what was done by CVRM and that “left a great deal of room for concerns to grow and spread unchecked” (Hollywood et al., 2019, p. 38). CVRM is a party-to-violence prediction system by design. In practice, it becomes a victim prediction system because the clearance rates for shootings in Chicago were constantly low. The Chicago Police Department’s lack of communicative transparency made it mistakenly conceived as a criminal prediction system. As a result, individuals noted by the system face unnecessary stigmatization, and the system lost social acceptability.
The subgroup identified by CVRM, for example, was individuals at risk of being victimized. For the program to succeed, the followed-up interventions must be guided towards reducing the likelihood for the subgroup to be victims of violence in the future. Without sufficient information to identify needed interventions for the targeted group, the program’s chance to succeed was slim.
According to the New Orleans Police Department (2011–2014), when the high-risk subgroups in the community are provided with the resources to improve, for example, their job prospects, there is indeed a significant reduction in homicide and gang-involved murders (Ferguson, 2017). The statics shows a significant difference in whether resources are implemented to increase the targeted individuals’ opportunities and chances to escape crime.
As Roberson et al. (2020, p. 55) put it, these programs “offer a venue for service providers from various sectors (police, education, addictions, social work, mental health, etc.) to regularly convene and discuss clients who meet a defined threshold of risk. The intent of these discussions is to formulate a plan of intervention that mobilizes multiple sectors, collaborating to provide services and support to the individual or families. To mitigate risk before harm occurs, [they] aim to connect clients to services within 24 to 48 h of a case being presented to the group.”
The programs “[hold] that violent crime can be dramatically reduced when law enforcement, community members, and social services providers join together to directly engage with street groups and gangs to clearly communicate: (1) a law enforcement message that any future violence will be met with clear, predictable, and certain consequences; (2) a moral message against violence by community representatives; and (3) an offer of help for those who want it” (von Ulmenstein and Sultan, 2011, p. 7). For details, see Kennedy and Friedrich (2014).
For example, as part of the efforts of the Office of Community Oriented Policing Services of the U.S. Department of Justice, law enforcement is encouraged to practice community policing by working with the communities they serve. It is noted that the role of law enforcement in the group violence intervention program is to identify the high-risk groups of exposure to violence, either as victims or as perpetrators, and to notify the targeted individuals who “are subjects of special law enforcement attention” (Kennedy & Friedrich, 2014, p. 26). The notification usually includes a custom legal assessment explaining the target’s legal exposure and information on social service resources available for the target and his/her families. Also see Braga et al. (2018) for a review of recent research on the effectiveness of this approach. They note that the existing theoretical literature and empirical evidence suggest that this approach generates noteworthy crime reductions.
References
Amnesty International. (2020). We sense trouble: Automated discrimination and mass surveillance in predictive policing in the Netherlands. Retrieved November 11, 2020, from https://www.amnesty.org/download/Documents/EUR3529712020ENGLISH.PDF
Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016). Machine bias. ProPublica. Retrieved November 11, 2020, from https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
Aougab, T., Ardila, F., Athreya, J., Goins, E., Hoffman, C., Kent, A., Khadjavi, L., O’Neil, C., Patel, P., & Wehrheim, K. (2020). Boycott collaboration with police. The Notices of the American Mathematical Society, 67(9), 1293.
Asher-Schapiro, A. (2020, June 25). California City bans predictive policing in U.S. First. Reuters. Retrieved November 11, 2020, from https://www.reuters.com/article/us-usa-police-tech-trfn/california-city-bans-predictive-policing-in-u-s-first-idUSKBN23V2XC
Babuta, A., & Oswald, M. (2020). Data analytics and algorithms in policing in England and Wales: Towards a new policy framework. Royal United Services Institute for Defence and Security Studies. Retrieved November 20, 2020, from https://rusi.org/sites/default/files/rusi_pub_165_2020_01_algorithmic_policing_babuta_final_web_copy.pdf.
Berthold, M. (2020, November 14). You can’t eliminate bias from machine learning, but you can pick your bias. VentureBeat. Retrieved November 20, 2020, from https://venturebeat.com/2020/11/14/you-cant-eliminate-bias-from-machine-learning-but-you-can-pick-your-bias/.
Braga, A. A., Weisburd, D., & Turchan, B. (2018). Focused deterrence strategies and crime control: An updated systematic review and meta-analysis of the empirical evidence. Criminology & Public Policy, 17(1), 205–250.
Castelvecchi, D. (2020). Mathematicians urge colleagues to boycott police work in wake of killings. Retrieved November 11, 2020, from https://www.nature.com/articles/d41586-020-01874-9
Couchman, H. (2019). Policing by machine, Liberty, LIB11. Retrieved November 20, 2020, from https://www.libertyhumanrights.org.uk/wp-content/uploads/2020/02/LIB-11-Predictive-Policing-Report-WEB.pdf
Coyle, D., & Weller, A. (2020). ‘Explaining’ machine learning reveals policy challenges. Science, 368(6498), 1433–1434.
Crawford, A., & Evans, K. (2012). Crime prevention and community safety. In A. Liebling, S. Maruna, & L. McAra (Eds.), The Oxford handbook of criminology (5th ed.). Oxford University Press.
Deto, R. (2020, August 25). Pittsburgh city council introduces police facial recognition, predictive policing ban. Pittsburgh City Paper. Retrieved November 11, 2020, from https://www.pghcitypaper.com/pittsburgh/pittsburgh-city-council-introduces-police-facial-recognition-predictive-policing-ban/Content?oid=17879052
Egbert, S., & Krasmann, S. (2020). Predictive policing: Not yet, but soon preemptive? Policing and Society, 8(3), 1–15.
Engqvist, T. (2020). The bias paradox: Are standpoint epistemologies self-contradictory? Episteme. https://doi.org/10.1017/epi.2020.21
Ensign, D., Friedler, S. A., Neville, S., Scheidegger, C., & Venkatasubramanian, S. (2018). Runaway feedback loops in predictive policing. Proceedings of Machine Learning Research, 81, 1–12.
Fajnzylber, P., Lederman, D., & Loayza, N. (2002). Inequality and violent crime. The Journal of Law and Economics, 45(1), 1–39.
Ferguson, A. G. (2017). The rise of big data policing: Surveillance, race, and the future of law enforcement. New York University Press.
Gilovich, T., Griffin, D., & Kahneman, D. (Eds.). (2002). Heuristics and biases: The psychology of intuitive judgment. Cambridge University Press.
Goel, S., Shroff, R., Skeem, J. L., & Slobogin, C. (2018). The accuracy, equity, and jurisprudence of criminal risk assessment. Retrieved November 11, 2020. https://doi.org/10.2139/ssrn.3306723
Hao, K. (2019, December 27). In 2020, let’s stop AI ethics-washing and actually do something. MIT Technology Review. Retrieved November 11, 2020, from https://www.technologyreview.com/2019/12/27/57/ai-ethics-washing-time-to-act/
Hardyns, W., & Rummens, A. (2018). Predictive policing as a new tool for law enforcement? Recent developments and challenges. European Journal on Criminal Policy and Research, 24, 201–218.
Haslanger, S. (2015). Distinguished lecture: Social structure, narrative and explanation. Canadian Journal of Philosophy, 45(1), 1–15.
Haslanger, S. (2017). Racism, ideology, and social movements. Res Philosophica, 94(1), 1–22.
Heaven, W. D. (2020, July 17). Predictive policing algorithms are racist. They need to be dismantled. MIT Technology Review. Retrieved November 11, 2020, from https://www.technologyreview.com/2020/07/17/1005396/predictive-policing-algorithms-racist-dismantled-machine-learning-bias-criminal-justice/
Hollywood, J. S., Mckay, K. N., Woods, D., & Agniel, D. (2019). Real-time crime centers in Chicago. Retrieved November 11, 2020, from https://www.rand.org/content/dam/rand/pubs/research_reports/RR3200/RR3242/RAND_RR3242.pdf
Hosein, A. O. (2018). Racial profiling and a reasonable sense of inferior political status. Journal of Political Philosophy, 26(3), e1–e20.
Hung, T.-W., & Yen, C.-P. (2020). On the person-based predictive policing of AI. Ethics and Information Technology. https://doi.org/10.1007/s10676-020-09539-x
Hung, T. W. (2020). A Preliminary Study of Normative Issues of AI Prediction. EurAmerica, 50(2), 229–252. https://doi.org/10.7015/JEAS.202006_50(2).0004
Hitachi Inc. (2019). Hitachi provides an AI environment in research on Kanagawa prefecture police’s crime and traffic accident prediction techniques. https://reurl.cc/lL6d2E. Accessed 16 Jan 2020.
Illinois Institute of Technology. (2019). Crime and victimization risk model (CVRM) fact sheet. Retrieved November 11, 2020, from https://home.chicagopolice.org/wp-content/uploads/2019/01/FACT-SHEET-Crime-and-Victimization-Risk-Model-1.pdf
Jenkins, R., & Purves, D. (2020). AI ethics and predictive policing: A roadmap for research. Retrieved November 11, 2020, from http://aipolicing.org/year-1-report.pdf
Kennedy, D. M., & Friedrich, M. (2014). Custom notifications: Individualized communication in the group violence intervention. Office of Community Oriented Policing Services. Retrieved November 20, 2020, from https://nnscommunities.org/wp-content/uploads/2017/10/GVI_Custom_Notifications_Guide.pdf
Kerr, I., & Earle, J. (2013). Prediction, preemption, presumption. Stanford Law Review Online, 65(September), 65–72.
Kleinberg, J., Lidwig, J., Mullainathan, S., & Sunstein, C. R. (2018). Discrimination in the age of algorithms. Journal of Legal Analysis, 10, 113–174.
Kroll, J. A. (2018). The fallacy of inscrutability. Philosophical Transactions of the Royal Society A, 376, 20180084.
Kroll, J. A., Huey, J., Barocas, S., Felten, E. W., Reidenberg, J. R., Robinson, D. G., & Harlan, Yu. (2017). Accountable algorithms. University of Pennsylvania Law Review, 165, 633–705.
Kusner, M. J., & Loftus, J. R. (2020). The long road to fairer algorithms. Nature, 578(7793), 34–36.
Lin, K.-H., Lee, H.-C., & Lin, T.-H. (2017). How does resilience matter? An empirical verification of the relationships between resilience and vulnerability. Natural Hazards, 88(2), 1229–1250.
Lin, T.-H. (2015). Governing natural disasters: State capacity, democracy, and human vulnerability. Social Forces, 93(3), 1267–1300.
Lin, Y. T., Hung, T. W., & Huang, L. T. L. (2020). Engineering equity: How AI can help reduce the harm of implicit bias. Philosophy & Technology. https://doi.org/10.1007/s13347-020-00406-7
Morley, J., Floridi, L., Kinsey, L., & Elhalal, A. (2019). From What to How: An initial review of publicly available AI ethics tools, methods and research to translate principles into practices. Science and Engineering Ethics, 26, 2141–2168.
Ohyama, T., & Amemiya, M. (2018). Applying crime prediction techniques to Japan: A comparison between risk terrain modeling and other methods. European Journal on Criminal Policy and Research, 24(4), 469–487.
Pearsall, B. (2010). Predictive policing: The future of law enforcement? NIJ Journal, 266, 16–19.
Perry, W. L., McInnis, B., Price, C. C., Smith, S. C., & Hollywood, J. S. (2013). The role of crime forecasting in law enforcement operations. RAND Corporation. Retrieved November 11, 2020, from https://www.rand.org/content/dam/rand/pubs/research_reports/RR200/RR233/RAND_RR233.pdf
Richardson, R., Schultz, J., & Crawford, K. (2019). Dirty data, bad predictions: How civil rights violations impact police data, predictive policing systems, and justice. New York University Law Review, 94, 192–233.
Roberson, K., Khoo, C., & Song, Y. (2020). To surveil and predict: A human rights analysis of algorithmic policing in Canada. Retrieved November 11, 2020, from https://citizenlab.ca/wp-content/uploads/2020/08/To-Surveil-and-Predict.pdf
Saunders, J., Hunt, P., & Hollywood, J. S. (2016). Predictions put into practice: A quasi-experimental evaluation of Chicago’s predictive policing pilot. Journal of Experimental Criminology, 12(3), 347–371.
Schuilenburg, M. (2021). Hysteria: Crime, media, and politics. Routledge.
Selbst, A. D. (2017). Disparate impact in big data policing. Georgia Law Review, 51(1), 109–195.
Shapiro, A. (2017). Reform predictive policing. Nature News, 541(7638), 458.
Sheehey, B. (2019). Algorithmic paranoia: The temporal governmentality of predictive policing. Ethics and Information Technology, 21, 49–58.
Skeem, J., & Lowenkamp, C. (2016). Risk, race, and recidivism: Predictive bias and disparate impact. Criminology, 54, 680–712.
Skeem, J., & Lowenkamp, C. (2020). Using algorithms to address trade-offs inherent in predicting recidivism. Behavioral Sciences & the Law, 38, 259–278.
Sloane, S., Baillargeon, R., & Premack, D. (2012). Do Infants have a sense of fairness? Psychological Science, 23(2), 196–204.
Susser, D. (2021). Predictive policing and the ethics of preemption. In B. Jones & E. Mendieta (Ed.), The ethics of policing: An interdisciplinary perspective. New York University Press.
Tsushima, M., & Hamai, K. (2015). Public Cooperation with the police in Japan: Testing the legitimacy model. Journal of Contemporary Criminal Justice, 31(2), 212–228.
von Ulmenstein, S., & Sultan, B. (2011). Group violence reduction strategy: Four case studies of swift and meaningful law enforcement responses. U.S. Department of Justice. Retrieved November 20, 2020, from https://nnscommunities.org/wp-content/uploads/2017/10/LE_Case_Studies.pdf
Wisconsin v. Loomis. (2016). Retrieved November 11, 2020, from https://www.wicourts.gov/sc/opinion/DisplayDocument.pdf?content=pdf&seqNo=171690
Zarsky, T. Z. (2013). Transparent predictions. University of Illinois Law Review, 2013(4), 1503–1570.
Zheng, R. (2018). Bias, structure, and injustice: A reply to Haslanger. Feminist Philosophy Quarterly. https://doi.org/10.5206/fpq/2018.1.4
Funding
Funding was provided by Ministry of Science and Technology, Taiwan (Grant No. 107-2410-H-001-101-MY3).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yen, CP., Hung, TW. Achieving Equity with Predictive Policing Algorithms: A Social Safety Net Perspective. Sci Eng Ethics 27, 36 (2021). https://doi.org/10.1007/s11948-021-00312-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11948-021-00312-x