Introduction

On July 14, 2025, Angela Lipps was babysitting in her Tennessee home when law enforcement officers arrived to arrest her for multiple fraud and property theft charges brought by a North Dakota State’s Attorney’s Office. Ms. Lipps was held in a county jail in Tennessee until October. She was then extradited to North Dakota and held in a Fargo jail until December 24, 2025, when a judge dismissed the charges. Ms. Lipps was subsequently released into a frigid North Dakota winter with only the summer clothing she had been wearing when she was originally arrested in Tennessee. 1Open this footnote Close this footnote 1 Michael Levenson, Woman Spent Five Months in Jail After A.I. Linked Her to Bank Fraud Case, N.Y. Times (Mar. 30, 2026), https://‌perma.cc/‌‌UL9E-QS77. …Open this footnote Close It turned out that Ms. Lipps had proof that she was in Tennessee when the bank fraud was taking place in North Dakota, and, in fact, had never been to the state nor knew anyone there. These facts would have been readily discovered through an investigation prior to arrest. But that sort of laborious and expensive retail policing was pushed aside in favor of reliance on ‌the rapid advice from an oracular computer program. This software selected Ms. Lipps through a facial recognition program that uses artificial intelligence (AI) techniques to algorithmically sort through billions of photographs scraped from the internet to find likely matches.

West Fargo police acknowledged the error, relying upon a facial recognition system that its manufacturer said was “designed to function as one tool within a broader investigative process,” but added that “independent corroboration by trained law enforcement professionals is required.” 2Open this footnote Close this footnote 2 Id. …Open this footnote Close The department’s chief acknowledged that they had modified the department’s AI policies since this incident. 3Open this footnote Close this footnote 3 Id. …Open this footnote Close It is not clear just how much independent corroboration took place between the initial AI-tool query and the issuance of a warrant for the extradition of Ms. Lipps to North Dakota—nor do the police know exactly how the software’s algorithms arrived at its conclusions. What is clear, however, is that the West Fargo police stopped short of specifying how their AI policies or use cases led to this outcome, acknowledging only that they made “a few errors” in this case, including their reliance on the use of AI tools by a “partner agency,” and apologizing “for any effect, or adverse effect, that [the incident] . . . had on trust in the community.” 4Open this footnote Close this footnote 4 Id.; Zoe Sottile, Police Used AI Facial Recognition to Arrest Tennessee Woman for Crimes Committed in a State She Says She’s Never Visited, CNN (Mar. 29, 2026, 6:00 AM ET), https://perma.cc/‌‌‌4PPJ-DRBW. …Open this footnote Close Police offered no apology to Ms. Lipps for her five-month incarceration, and she had to rely on donations to make her way to Chicago.

Unfortunately, Ms. Lipps’s story is not unique. Examples of law enforcement’s use of AI technologies resulting in wrongful arrests occur with unnerving frequency. 5Open this footnote Close this footnote 5 See, e.g., Daniel Wu, ‘That Wasn’t Me’: How Facial Recognition Led to a Woman Being Jailed for 6 Months, Wash. Post (Apr. 14, 2026), https://perma.cc/‌‌3ALG-KGJX (describing police use of facial recognition technology to identify and charge a suspect despite contrary evidence); Joe Wilkins, Police Arrest Man With Almost Zero Resemblance to Actual Perpetrator Because AI Told Them To, Futurism (Aug. 29, 2025, 4:29 PM ET), https://perma.cc/‌‌WV8R-H988 (describing the New York Police Department’s use of facial recognition software leading to the misidentification of an African American man); Justin Schecker, Orlando Police Wrongful Arrest Fits Pattern of Similar Cases Using Facial Recognition, Attorney Says, WESH 2 News (Mar. 30, 2026, 9:17 PM EDT), https://perma.cc/‌‌‌5XJX-YXX7 (describing an incident of the Orlando Police mistakenly arresting a man after a facial recognition match); Douglas MacMillan, David Ovalle & Aaron Schaffer, Arrested by AI: Police Ignore Standards After Facial Recognition Matches, Wash. Post (Jan. 13, 2025), https://perma.cc/‌‌JSS9-V4D9 (detailing the police’s continued use of facial recognition technologies even after flaws in their use became known). Mistakes made by commercial use of AI technologies can also lead to arrests and investigations. See Rob Thubron, Target Puts Customers on the Hook for AI Shopping Assistant Errors, TechSpot (Apr. 6, 2026, 9:43 AM ET), https://perma.cc/‌‌8P8E-YZ4U (describing how companies like Target use contract terms and conditions to put the burden of AI-system mistakes on consumers); see also AI Retail Surveillance & False Shoplifting Accusations: Legal Guide, Humanoid Liab. (Dec. 9, 2025), https://perma.cc/‌‌P4ZU-ZGZB (outlining how retailers have been using AI tools to identify shoplifters with many false positives, leading to “public humiliation, wrongful detention, false arrests, and permanent damage to reputations”). …Open this footnote Close Much has been written over the past thirty years regarding the potential risks inherent in law enforcement’s use of technology to enhance policing effectiveness and efficiency, and questions of bias, legitimacy, militarization, and privacy remain highly relevant. 6Open this footnote Close this footnote 6 See Richard V. Ericson & Kevin D. Haggerty, Policing the Risk Society 5 (1997) (illustrating how “[m]ost of the crime-related knowledge produced by the police is disseminated to other institutions (for example, those concerned with health, insurance, public welfare, financial matters, and education) for their risk management needs, rather than used for criminal prosecution and punishment”). See generally Rashida Richardson, Jason M. Schultz & Kate Crawford, Dirty Data, Bad Predictions: How Civil Rights Violations Impact Police Data, Predictive Policing Systems, and Justice, 94 N.Y.U. L. Rev. Online 15 (2019) (criticizing use of predictive policing technologies which result in bias and unlawful practices); Peter K. Manning, Information Technologies and the Police, Crime & Just., no. 15, 1992, at 349, 383-88 (detailing police use of technology at various levels and how those uses affect police culture); Elizabeth E. Joh, Reckless Automation in Policing, Berkeley Tech. L.J. Commentaries, July 11, 2022, at 118, 120-21, https://perma.cc/‌3RQA-BU2P (describing how police use of automation technology can reduce law enforcement accountability). But see Elizabeth E. Joh, Discretionless Policing: Technology and the Fourth Amendment, 95 Calif. L. Rev. 199, 199-200 (2007) (recommending technological approaches to reducing discretion in policing). …Open this footnote Close But the scope, scale, and speed of the adoption and use of AI technology raises new—and more urgent—concerns. As the West Fargo Police Department example illustrates, use of AI tools tends to obfuscate, rather than illuminate, knowledge of the problems we task them to solve. Moreover, it encourages users to defer to these tools as a heuristic replacement for their own professional judgment and leads people to ascribe an elevated level of expertise, and even agency, to the software as an unbiased observer. This combination is a recipe for reduced accountability for the inevitable errors these tools will produce—an especially acute problem in law enforcement.

This Essay argues that law enforcement’s uses of AI technologies can become a means of avoiding police accountability, both consciously and unconsciously, since reliance on these technologies can reduce internal and external understanding of relevant facts, play into our natural inclinations toward automation bias, and encourage succumbing to the convenience of automation over human judgment. In Part I, I discuss “agnotology,” the study of the deliberate production of ignorance, 7Open this footnote Close this footnote 7 The definition of agnotology I will be relying on in this Essay is summed up by Amit Ray and Michael Nolan, who describe it as “the study of the deliberate cultural production of ignorance” and note it is “a concept applied to modern scientific endeavors, particularly with regard to the science emerging from for-profit, proprietary entities—but importantly, not limited to them.” Amit Ray & Michael Nolan, Artificial Ignorance: Understanding the Role of AI in Modern Agnotology, First Monday (Aug. 4, 2025), https://perma.cc/‌UB3Y-PBPL. …Open this footnote Close and how AI technologies can manufacture ignorance rather than knowledge, often in unexpected ways. In Part II, I examine the human predisposition toward automation bias and how that tendency is exacerbated by the characteristics of AI tools. Finally, in Part III, I describe how these characteristics of AI technologies, along with our readiness to accept AI output as a “view from nowhere,” lead to the degradation of accountability for police errors—even when those errors leave very real harms in their wake—and put far too much trust in the judgment of AI companies.

I. AI and the Problem of Knowledge

The word technology is surprisingly difficult to define, but its close relationship with the older and broader term technics—which describes the mechanical and applied arts— provides a basis for our purposes. 8Open this footnote Close this footnote 8 See Technic, Oxford English Dictionary, https://perma.cc/‌JQX7-3EMX (archived Aug. 12, 2026). …Open this footnote Close The history of civilization can be framed by the use of technics to overcome our rather limited faculties, and enable humanity’s expansion in ways that would have been otherwise impossible. 9Open this footnote Close this footnote 9 See Lewis Mumford, Technics and Civilization 3-7 (Univ. Chi. Press 2010) (1934) (describing the historical relationship between humanity and technology). …Open this footnote Close N. Bruce Hannay and Robert E. McGinn have observed that “the basic function of technology is the expansion of the realm of practical human possibility.” 10Open this footnote Close this footnote 10 N. Bruce Hannay & Robert E. McGinn, The Anatomy of Modern Technology: Prolegomenon to an Improved Public Policy for the Social Management of Technology, Dædalus, Winter 1980, at 25, 35. …Open this footnote Close The tools we create to expand our realms of possibility, however, bring with them the worldviews of their creators and have effects on the way we live, the ways in which we relate to one another, and the ways we relate to our machines. This fact makes it necessary to view technology through the lenses of law and political philosophy. 11Open this footnote Close this footnote 11 See, e.g., Langdon Winner, The Whale and the Reactor: A Search for Limits in an Age of High Technology 4 (1986) (“The basic task for a philosophy of technology is to examine critically the nature and significance of artificial aids to human activity.”). …Open this footnote Close

The advent of the early computer—first mechanical, then digital—brought with it the promise of new realms of efficiency and productivity, where businesses, financial institutions, and governments could benefit from the computer’s new calculation and data-processing capabilities and, through these affordances, operate in groundbreaking ways. Any activity whose functions depended on the maintenance and rapid analysis of large amounts of information could profit from this technology. The activities associated with law enforcement fit well within this model, as their investigation and reporting roles were built on information management. The Johnson administration reported on this alignment through its Commission on Law Enforcement and the Administration of Justice, which recommended allocating federal funding for technology in policing in 1967. 12Open this footnote Close this footnote 12 President’s Comm’n on L. Enf’t & Admin. Just., The Challenge of Crime in a Free Society 251-55 (1967). …Open this footnote Close

A. Information Automation

Technology sales teams quickly lined up to sell their products to police departments, ramping up their efforts when spending for the Vietnam War began to wind down, and the domestic market became even more important to these companies’ bottom lines. 13Open this footnote Close this footnote 13 See generally Jonathan Simon, Sacrificing Private Ryan: The Military Model and the New Penology, in Militarizing the American Criminal Justice System: The Changing Roles of the Armed Forces and the Police 105-19 (Peter B. Kraska ed., 2001) (describing the cultural and psychological militarization of civilian police agencies through the adoption of militarized philosophies); Peter B. Kraska, Playing War: Masculinity, Militarism, and Their Real-World Consequences, in Militarizing the American Criminal Justice System: The Changing Roles of the Armed Forces and the Police 141, 141-57 (Peter B. Kraska ed., 2001) (detailing the transferal of military technologies to civilian police departments in the post-Vietnam War years). …Open this footnote Close Despite some efficiency gains in data management from these early installations, putting computers in police departments did not automatically lead to widespread innovation in everyday policing techniques. 14Open this footnote Close this footnote 14 See Kent W. Colton, The Impact and Use of Computer Technology by the Police, 22 Commc’ns ACM 10, 11-12 (1979). …Open this footnote Close By the mid-1970s, however, police agencies began to feel significant political pressure to address the “spiral deeper into decline” that came with the economic slowdown of a deindustrializing America. 15Open this footnote Close this footnote 15 Wesley G. Skogan, Disorder and Decline: Crime and the Spiral of Decay in American Neighborhoods 13 (1990); see William J. Chambliss, Power, Politics, And Crime 21-24 (2001) (describing an increased political focus on “tough on crime” legislation based on the promotion of fear-based messaging); David Seidman & Michael Couzens, Getting the Crime Rate Down: Political Pressure and Crime Reporting, 8 L. & Soc’y Rev. 457, 469, 474-76 (1974) (identifying political pressure in the 1960s and early 1970s as having perverse effects on the accuracy of crime reporting mechanisms); Robert J. Sampson & William Julius Wilson, Toward a Theory of Race, Crime, and Urban Inequality, in Race, Crime, and Justice: A Reader 177, 179-80 (Shaun L. Gabbidon & Helen Taylor Greene eds., 2005) (responding to assertions regarding race and violent crime and identifying economic deprivation as a significant indicator of crime rates generally); George L. Kelling & William J. Bratton, Declining Crime Rates: Insiders’ Views of the New York City Story, 88 J. Crim. L. & Criminology 1217, 1218-20 (1998) (describing the “broken windows” theory that holds that a broken window or other minor quality of life degradations, if left unattended, will inevitably lead to more disorder and crime). …Open this footnote Close Experts recommended a new approach to policing that argued for a bottom-up approach, aggressively addressing petty crimes and community disorder under the theory that more serious crimes flourished where these elements were present. 16Open this footnote Close this footnote 16 George L. Kelling & James Q. Wilson, Broken Windows: The Police and Neighborhood Safety, Atlantic (Mar. 1982), https://perma.cc/‌PV88-BE2C. …Open this footnote Close This approach demanded a reorganization of police departments around a data-centric philosophy that would take in the vast quantities of data generated by everyday police activities, analyze that information, and provide insights that could significantly increase a police department’s effectiveness for less money—attractive prospects to local taxpayers. By the 1990s, computers—now cheaper, smaller, and orders of magnitude more powerful than the hulking mainframes of the 1960s—became essential equipment for every law-enforcement agency. 17Open this footnote Close this footnote 17 See Jeffrey L. Vagle, Tightening the OODA Loop: Police Militarization, Race, and Algorithmic Surveillance, 22 Mich. J. Race & L. 101, 114-16, 119-20 (2016). …Open this footnote Close

Today’s AI-technology offerings address exactly this information-based approach to policing. If software can make sense of the mountain of information now available to law enforcement to reduce crime and costs, it would be foolish not to use it. Patterns exist in data, and while humans might not see them, a powerful AI-based application might—thereby giving police officers the ability to predictively patrol areas that have been algorithmically identified as particularly at-risk for crimes, and preempt or intervene before those crimes occur. Predictive policing has long been a dream of law enforcement, with various experiments and approaches applied over the last century. 18Open this footnote Close this footnote 18 See Andrew Guthrie Ferguson, Policing Predictive Policing, 94 Wash. U. L. Rev. 1109, 1112, 1115, 1117-19, 1123-24 (2017) (examining and criticizing the use of predictive policing technologies, especially their use to reduce transparency and shield accountability). …Open this footnote Close

B. AI as Truth Arbiter

If we assume that, by at least some metrics, increasing policing efficiency and effectiveness through technological tools is worthwhile, this process must include a good-faith production of these tools’ outputs and outcomes. The analytical steps taken to generate these outputs and outcomes should be both accessible and comprehensible to the public. Without this information, questions of fairness, efficacy, future directions for research and development, and due process would remain unanswered. We would be left only with an unfounded faith that such systems were yielding—or at least striving to yield—the best of all possible worlds. 19Open this footnote Close this footnote 19 For more in-depth discussions of these issues, see generally Helen Nissenbaum, Computing and Accountability, Commc’ns ACM, Jan. 1994 (illustrating the accountability problems that arise when technologies augment or replace human judgment); Andrew Guthrie Ferguson, Generative Suspicion and the Risks of AI-Assisted Police Reports, 120 Nw. U. L. Rev. 299 (2025) (describing the uses of AI technologies by police departments to generate official reports and the multiple accountability and due process problems that accompany these uses); Ian T. Adams, Matt Barter, Kyle McLean, Hunter Boehme & Irick A. Geary, No Man’s Hand: Artificial Intelligence Does Not Improve Police Report Writing Speed, CrimRxiv (Sept. 11, 2024), https://perma.cc/‌2YAS-4PDT (empirical study finding police use of large language models does not improve reporting efficiency); and Mareile Kaufmann, AI in Policing and Law Enforcement, in Handbook on Public Policy and Artificial Intelligence 295, 295 (Regine Paul, Emma Carmel & Jennifer Cobbe eds., 2024) (outlining debates on use of technologies in predictive policing). For a literature review regarding fairness, accountability, and transparency in AI, see Tashil Rama & Tania Prinsloo, Fairness, Accountability, Transparency, and Ethics (FATE) in Artificial Intelligence Creation: A Systematic Literature Review, 2 Digital Sols. for Env’t & Econ. Dev.: Select Procs. ICEIL 153, 157-64 (Balvinder Shukla, B.K. Murthy, Nitasha Hasteer, Sumeet Gupta & Diptiranjan Mahapatra eds., 2024). …Open this footnote Close

Herbert Simon wrote that technology could be thought of as existing at a meeting point between its “inner” environment and an “outer” environment. How—and how well—a technology works will depend on its functioning at this external interface. 20Open this footnote Close this footnote 20 Herbert A. Simon, The Sciences of the Artificial 6 (3d ed. 1996). …Open this footnote Close Using this understanding of a technology, we can make predictions about its behavior with little understanding of its inner environment. For example, if we discern the ability of a camera to withstand the extremities of space travel, we can decide whether that device belongs on a Mars probe or if it is better suited for vacation snapshots. But there are also contexts within which we may want or need detailed knowledge of a technology’s inner design, organization, or functions. We require this kind of inner information, for example, in patent applications, both for broad reasons of public policy and, more practically, as a means to discern and differentiate an inventor’s claims. Similarly, accountability requires an understanding of the inner environment of technological systems used in ways that may result in real-world harms, such as the design and implementation of anti-lock braking mechanisms, aircraft avionics, and banking systems. Understanding these technologies’ inner environments helps assign culpability for injuries caused by these technologies as well as develop a knowledge of the actual causes of those injuries to avoid repetition of these problems and improve future performance. This desire for accountability for harms caused by poorly implemented or faulty technologies is as old as human civilization. 21Open this footnote Close this footnote 21 See, e.g., The Code of Hammurabi, King of Babylon § 229-33, at 45-47 (Robert Francis Harper trans., 2d ed. 1904) (c. 2250 B.C.E.) (ancient legal code containing multiple rules and punishments, including, for example, penalties for builders whose work is flawed and results in loss of property or personal injury). …Open this footnote Close Thus, to fully evaluate the purpose, efficacy, and soundness of a technology, knowledge of both the internal and external environments of the tool becomes necessary.

AI technologies tend to frustrate this process, however, as they often obfuscate both internal and external knowledge of their systems. Internally, the rapid deployment of these systems; the opacity of their data collection practices, design, and structural characteristics; and claims of the unknowable quality of the stochastic models that AI systems are built upon mean that we have little to no idea how these technologies arrive at a particular decision or produce specific output. 22Open this footnote Close this footnote 22 Ray & Nolan, supra note 7; Jennifer L. Croissant, Agnotology: Ignorance and Absence or Towards a Sociology of Things That Aren’t There, 28 Soc. Epistemology 4, 15-16 (2014) (describing how some representations of information and methods of organizing data can paradoxically obfuscate information, making it invisible or unintelligible). …Open this footnote Close Not only does this algorithmic ignorance mean fewer ways to understand and control processes and correct for errors or biases along the way, but it also puts far too much power into the hands of the companies building these AI technologies, forcing a deference to their judgment on the internal effects and safety of their tools. This also provides AI developers with exclusive access to knowledge of user behaviors and methods that can be profitably exploited in their research pipeline, leaving the public with an incomplete internal knowledge picture—which they have little to no ability to audit—that has been carefully curated by the AI tool developer. 23Open this footnote Close this footnote 23 See generally Alondra Nelson, Algorithmic Agnotology: On AI, Ignorance, and Power, U. Wash. Tech. Pol’y Lab (Apr. 3, 2025), https://perma.cc/‌3W3E-QL8G (discussing how AI companies strategically blur the distinction between epistemic and stochastic uncertainty to lessen accountability to users). …Open this footnote Close

The development of external knowledge also leads to AI technologies replacing or foreclosing the possibility of broad data search and analysis, sealing this external knowledge in a way that hinders the qualitatively useful leverage of context and human judgment. 24Open this footnote Close this footnote 24 See Nora Freya Lindemann, Chatbots, Search Engines, and the Sealing of Knowledges, 40 AI & Soc’y 5063, 5063-64 (2025). …Open this footnote Close AI tools are deployed to give their users what is conveyed as the “best” answer to their queries, as the tools are designed to reduce user uncertainty. The AI technology’s message is essentially that “[t]here’s nothing else to be said” on the matter, which will frequently be accepted as gospel by users trained to trust these systems. 25Open this footnote Close this footnote 25 Martin Potthast, Matthias Hagen & Benno Stein, The Dilemma of the Direct Answer, ACM SIGIR F., June 2020, at 1, 2-4, 8. …Open this footnote Close Understanding how well a tool is suited to a particular problem space depends heavily on the spectrum of external knowledge we can obtain from that tool; fewer ways to gain that external knowledge means fewer ways to gauge the tool’s performance, which is especially troublesome when these systems are used as information sources for law-enforcement agencies. As these tools become more widely used in the policing context, the sealed external knowledge they provide becomes grounded truth, making it increasingly difficult to assess whether the tools are truly suited for their purpose or just appear that way. 26Open this footnote Close this footnote 26 See Bent Flyvbjerg, AI as Artificial Ignorance, Project Leadership & Soc’y, Dec. 2025, at 1, 2-3 (describing the output of one AI-powered chatbot as a “bullshit generator” in the Frankfurtian sense of the term). …Open this footnote Close At a minimum, policies regarding law enforcement use of AI technologies require both internal and external knowledge as evidence to identify potential accountability threats and risks of permanent harm. 27Open this footnote Close this footnote 27 See Rishi Bommasani et al., Advancing Science- and Evidence-Based AI Policy, 389 Science 459, 459-60 (2025) (arguing for AI policies that are built on scientific evidence and take into account the broadest range of policy outcomes possible). …Open this footnote Close

When deployed as part of the law-enforcement process, this deliberate manufacturing of ignorance can create real problems for the protection of civil liberties. Police powers to detain, arrest, or even use lethal force based on the output from AI systems that have sealed off internal and external knowledge of those technologies leave civil society at the mercy of the tools themselves, as well as their creators and users. Consider, for example, Angela Lipps’s arrest based on the West Fargo Police Department’s use of facial recognition technology, which introduced this Essay. Despite the fact that the AI system that identified Ms. Lipps generated a false positive—a fact that some basic police work could have uncovered at the beginning of the investigative process—the police officials involved did not accept responsibility for their errors. 28Open this footnote Close this footnote 28 See supra note 1 and accompanying text. …Open this footnote Close In fact, even if the West Fargo Police had intended to investigate the problem with the facial recognition technology that led to their error, it would be unlikely that they would be able to fix the issue, as the internals of AI systems almost always present as black boxes to their users. Police departments like West Fargo’s can thus be indemnified through the good-faith use of AI technologies.

II. Trusting the Machine

We have a complicated relationship with machines and automation. We can be skeptical of new or unfamiliar technologies, but once they have been established within our environment, we grant these systems a high degree of trust. We even believe their output when it clearly contradicts information available from other sources, including our own senses. 29Open this footnote Close this footnote 29 See Kathleen L. Mosier & Linda J. Skitka, Automation Use and Automation Bias, 43 Procs. Hum. Factors & Ergonomics Soc’y Ann. Meeting 344, 344-45 (1999) (explaining the concept of automation bias, where people using automated systems are often susceptible to reduced vigilance and cognitive offloading, replacing human bases for judgment with recommendations from automated systems). …Open this footnote Close This tendency toward automation bias—the preference for information from automated systems over other sources—leads to a lack of vigilance, a deliberate ignorance of conflicting information, and cognitive surrender. We devalue human judgment and deliberation in favor of lightly scrutinized AI system outputs. 30Open this footnote Close this footnote 30 Steven D. Shaw & Gideon Nave, Thinking—Fast, Slow, and Artificial: How AI is Reshaping Human Reasoning and the Rise of Cognitive Surrender 3 (Wharton Sch., Working Paper No. 20260111, 2026), https://perma.cc/‌26MZ-XY6X. …Open this footnote Close Harms from our susceptibility to automation bias are well-documented in the commercial airline industry, where automated systems, designed to reduce the cognitive workload of pilots, can become substitutes for their professional judgment, leading to deadly consequences. 31Open this footnote Close this footnote 31 See Kathleen L. Mosier, Linda J. Skitka, Susan Heers & Mark Burdick, Automation Bias: Decision Making and Performance in High-Tech Cockpits, 8 Int’l. J. Aerospace Psych. 47, 51 (1998). …Open this footnote Close This bias is also prevalent in situations where users have limited knowledge or visibility into the tools they are required to rely on as part of their duties, a common occurrence within law-enforcement agencies where automated information and data processing have become the keys to efficient policing. 32Open this footnote Close this footnote 32 See Sarah Valentine, Impoverished Algorithms: Misguided Governments, Flawed Technologies, and Social Control, 46 Fordham Urb. L.J. 364, 394-95 (2019). …Open this footnote Close Because of our natural tendency toward heuristics in decisionmaking, automated solutions can present significant risks, especially when those systems become mandatory parts of those decisionmaking processes with real-world effects.

Our complicated relationship with machines does not end there. Users of AI technologies can even attribute moral agency or general intelligence to these systems. 33Open this footnote Close this footnote 33 Eyal Aharoni, Sharlene Fernandes, Daniel J. Brady, Caelan Alexander, Michael Criner, Kara Queen, Javier Rando, Eddy Nahmias & Victor Crespo, Attributions Toward Artificial Agents in a Modified Moral Turing Test, Nature: Sci. Reps., no. 14, 2024, at 1, 1-2, https://perma.cc/‌P6ML-J4CX. …Open this footnote Close This tendency is partially due to persistent marketing efforts on the part of AI companies combined with public statements by company leaders promoting their AI products as general purpose technologies positioned to rival the steam engine and electricity in their capacity to change the world forever. 34Open this footnote Close this footnote 34 Beth Stackpole, The Impact of Generative AI as a General-Purpose Technology, MIT Sloan Sch. Mgmt. (Aug. 6, 2024), https://perma.cc/‌5C8V-XX5Q. …Open this footnote Close A significant portion of user perception of AI technologies is based on this messaging, so it is not surprising when people attribute general intelligence to these systems, despite the fact that the AI company leaders themselves have admitted that much of this grandstanding is just marketing hype. 35Open this footnote Close this footnote 35 See Sarah Perkel, Anthropic CEO Says AGI is a Marketing Term and the Next AI Milestone Will Be Like a ‘Country of Geniuses in a Data Center’, Bus. Insider (Jan. 22, 2025, 10:13 AM GMT), https://perma.cc/‌SZ9C-JWVW. In fact, the term “artificial intelligence” itself was originally coined as a marketing term by academics in the 1950s to generate interest in their summer workshop on computers. Emily M. Bender & Alex Hanna, The AI Con: How to Fight Big Tech’s Hype and Create the Future We Want 11 (2025). …Open this footnote Close And because AI technologies such as large language models (LLMs) are designed to mimic or give the appearance of human understanding, we are also inclined to attribute moral characteristics to these systems. One study devised a Moral Turing Test, a variation of the classical Turing Test, which a computer can “pass” if humans cannot differentiate between its responses and a fellow human’s. The Moral Turing Test asks people to distinguish human moral responses from a computer’s, and in the study, subjects rated the AI system’s moral reasoning as superior to a human’s in areas such as virtuousness, intelligence, and trustworthiness. 36Open this footnote Close this footnote 36 Aharoni et al., supra note 33, at 1-2. …Open this footnote Close It was not that the respondents could not tell which responses belonged to a human and which to the AI system; rather, because the respondents thought that certain responses came from an AI system, they automatically attributed higher moral status to those responses. A police officer who grows to view an AI tool’s judgment as equal or superior to human moral intelligence will be more inclined to accept that tool’s output as a grounded truth.

Our readiness to ascribe intelligence—a controversial term in itself—to AI technologies like LLMs is due in large part to our natural inclination toward treating language use as the expression of communicative intent. Language is apparently so deeply embedded in our cognitive selves that, as children, we are able to grasp the rules of language in an astonishingly short amount of time, and when we hear speech or see text, we interpret it reflexively, as if without conscious thought. 37Open this footnote Close this footnote 37 See generally Noam Chomsky, Aspects of the Theory of Syntax 47-59 (1965) (describing a theory of syntax in language and linguistic frameworks); Daniel C. Dennett, The Role of Language in Intelligence, in What is Intelligence? 161, 161-178 (Jean Khalfa ed., 1994) (explaining how cognition is inextricably linked to the evolutionary development of language). …Open this footnote Close Because of the centrality of language to human cognition, computers that can take natural language as input and respond with natural language output have long been a significant goal for AI researchers. 38Open this footnote Close this footnote 38 See, e.g., Ira Goldstein & Seymour Papert, Artificial Intelligence, Language, and the Study of Knowledge, 1 Cognitive Sci. 84, 85-87 (1977) (describing relationship between AI and language and the subsequent need to improve machine comprehension of human languages). …Open this footnote Close When LLMs became feasible due to significant improvements in computing power and storage capacities in the 2000s, their ability to read and respond to natural language queries was often astonishing. These advances were increasingly seen and promoted as proof that AI systems were not mere software programs, but were thinking, learning, and understanding in much the same way we do, despite a majority of experts in fields like linguistics, cognitive science, and philosophy of mind saying otherwise. 39Open this footnote Close this footnote 39 See, e.g., Veena D. Dwivedi, A Neuroscientist Explains Why It’s Impossible for AI to ‘Understand’ Language, Conversation (June 5, 2025, 9:13 AM EDT), https://perma.cc/‌94RL-669T; Erik J. Larson, The Myth of Artificial Intelligence: Why Computers can’t Think the Way We Do 50-59 (2021) (illustrating how artificial intelligence technologies do not “think” in the way humans do); Daniel C. Dennett, Can Machines Think?, in Alan Turing: Life and Legacy of a Great Thinker 295, 295 (Christof Teuscher ed., 2004) (examining what we mean by “thinking” and what that means for artificial intelligence). See generally Bender & Hanna, supra note 35, at 21-41 (taking a skeptical look at the colloquial use of the term “think” when it comes to AI technologies). …Open this footnote Close Our biological eagerness to equate language with cognition can blind us to the actual work being done by language-based AI technologies.

Further, there is a tendency for AI technologies to act as “yes, and” machines, reflexively agreeing with or flattering users, or validating their prior beliefs. 40Open this footnote Close this footnote 40 See Myra Cheng, Cinoo Lee, Pranav Khadpe, Sunny Yu, Dyllan Han & Dan Jurafsky, Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence, 391 Science 1348, 1349 (2026). …Open this footnote Close This may increase user trust in these systems to the point where they increasingly trust the AI systems’ output to justify their decisions. 41Open this footnote Close this footnote 41 Id. …Open this footnote Close The sources of this problem lie both with the creators of these AI systems as well as in our human frailty. AI companies are facing enormous pressure from their investors to realize returns on those investments, and these firms have been turning that pressure on their customers in a quest to increase AI technology adoption in any way they can. 42Open this footnote Close this footnote 42 Lee Ying Shan, Private Credit Worries Resurface in $3 Trillion Market as AI Pressures Software Firms, CNBC (last updated Feb. 9, 2026, 2:01 AM EST), https://perma.cc/‌9S3V-Z9A3. …Open this footnote Close AI systems that maximize user engagement are therefore more attractive to these companies, so they are incentivized to tune these systems to agree with and even flatter their human customers. We often respond to such social sycophancy from these systems as if they were trusted advisors or therapists, looking out for our well-being while providing an unbiased critique of our thoughts and observations. This combination poses obvious risks to vulnerable populations suffering from depression or self-delusion, and has resulted in many documented tragedies, including mass shootings, self-harm, and suicides. 43Open this footnote Close this footnote 43 See, e.g., Mark Follman, The Chilling Role of ChatGPT in Mass Shootings and Other Violence, Mother Jones (July/Aug. 2026), https://perma.cc/‌G8JR-Y5WW (discussing how chatbots and other AI technologies have been used by those committing mass shootings); John Sanford, Why AI Companions and Young People Can Make for a Dangerous Mix, Stan. Med. (Aug. 27, 2025), https://perma.cc/‌4NZU-TJ7H (outlining the mental health risks of chatbot AI technologies for young people); Laura Kuenssberg, ‘A Predator in Your Home’: Mothers Say Chatbots Encouraged Their Sons to Kill Themselves, BBC (Nov. 8, 2025), https://perma.cc/‌4X3B-NKGU (detailing how chatbot AI technologies have been used by people to support suicidal ideations); Rachel Myrow, Google Updates Suicide, Self-Harm Safeguards in Gemini as AI Lawsuits Mount, KQED (Apr. 7, 2026, 7:04 PM PT), https://perma.cc/‌6R9S-2YNQ (explaining how one company is trying to prevent chatbot use that encourages suicide and ideations of self-harm). …Open this footnote Close

Even trained professionals are not immune to the social sycophancy of automated decisionmaking systems. Policing organizations are increasingly looking to LLMs to streamline areas such as report writing and officer questions regarding the law or departmental policies. 44Open this footnote Close this footnote 44 Ferguson, supra note 19, at 302-03, 306-10. …Open this footnote Close If the LLM’s programmed behavior is to provide output in a manner that tends to support an officer’s or department’s desired outcome, those users will tend to rely on these tools over their own or other human experts’ judgment. Even the companies marketing these AI technologies to police departments acknowledge this tendency, warning their users that human oversight should always be present, even though human users will often fall into patterns of cognitive surrender when using automated AI tools. 45Open this footnote Close this footnote 45 See The Definitive Guide to Using AI in Policing, Axon, https://perma.cc/‌6Z3E-PXDP (archived July 24, 2026). …Open this footnote Close

III. Agency, Perspective, and Accountability

Accountability is a necessary component of law enforcement in a democracy. 46Open this footnote Close this footnote 46 See Barry Friedman & Maria Ponomarenko, Democratic Policing, 90 NYU L. Rev. 1827, 1835 (2015) (“There are two core values to American democracy: democratic accountability and the rule of law.”). …Open this footnote Close General concerns about computer technologies eroding our ability to apportion accountability for harms are nothing new. Because so much of everyday life depends on computers and networked systems, the characteristics of these systems and their uses can often frustrate efforts to hold someone accountable for injuries. Exacerbating factors include an increased attenuation between activities and their resulting harms, the fact that defect-free computing is an impossibility, the placing of all blame on the machine (and thus not on its creators), and the conscious separation between software ownership and liability under U.S. technology policy. 47Open this footnote Close this footnote 47 See Nissenbaum, supra note 19, at 75-78. …Open this footnote Close

A. The Problem of Technological Determinism

There exists a school of thought that posits technology as a primary agent of change in human civilization, a worldview that has taken on increased valence as technologies like interconnected computers have become increasingly sophisticated and widespread throughout society, and occurs “when [a technology’s] introduction into the mainstream requires a systematic change to the law or legal institutions in order to reproduce, or if necessary displace, an existing balance of values.” 48Open this footnote Close this footnote 48 Ryan Calo, Robotics and the Lessons of Cyberlaw, 103 Calif. L. Rev. 513, 552 (2015). …Open this footnote Close This technological determinism is a controversial philosophy, as it demotes the effects of doctrine, reasoning, policies, and legal, social, and political theory to mere servants of technological advancement. 49Open this footnote Close this footnote 49 See Meg Leta Jones, Does Technology Drive Law? The Dilemma of Technological Exceptionalism in Cyberlaw, 2018 U. Ill. J.L. Tech. & Pol’y 249, 255-57 (2018) (arguing that technological determinism is a flawed approach to social, political, and legal policy creation and analysis). …Open this footnote Close A technological deterministic approach to public policies regarding law enforcement use of AI technologies would be a harmful example of the tail wagging the dog.

We must resist the tendency toward technological determinism and the idea that technological progress follows an arc that yields predicable societal results. Any technology is the result of a series of decisions made by designers, developers, and distributors to create, build, and sell a product, often in ways that hide much of the complexity of that technology to make it more useful to their end user. AI tools follow this model, with the promise that their systems will take over many of our cognitive tasks which we have decided are too mundane, too annoying, too complex, or too time-consuming to take on ourselves. This model prioritizes speed and convenience over careful consideration and difficult decisionmaking and is geared toward the goal of finding the correct (or at least close enough to correct) information in the shortest time possible. The philosophy behind this model assumes that the goal of any information analysis or decision-supporting task is to find the one “best” answer that human domain experts might give if they had enormous quantities of time and resources. The acceptance of this model across multiple contexts has led to a broad assumption that the responses we receive from AI tools are objective and represent an unbiased and dispassionate analysis of all available data—essentially a “view from nowhere.” 50Open this footnote Close this footnote 50 See Sarah Marie Stitzlein, Replacing the ‘View from Nowhere’: A Pragmatist-Feminist Science Classroom, Elec. J. Rsch. Sci. & Math Educ., Dec. 2004, at 1, 5. …Open this footnote Close

B. AI Systems Do Not Have Agency

But we know that AI technologies are anything but unbiased oracles of truth. Their output takes on a uniformly authoritative tone, even when miscounting the number of “r” characters in “strawberry” or failing to differentiate one Black face from another. 51Open this footnote Close this footnote 51 Kit Eaton, How Many R’s in ‘Strawberry’? This AI Doesn’t Know, Inc. (Aug. 28, 2024), https://perma.cc/‌XHQ8-LZDM; Thaddeus L. Johnson & Natasha N. Johnson, Police Facial Recognition Technology Can’t Tell Black People Apart, Sci. Am. (May 18, 2023), https://perma.cc/‌9TG2-SYYD. …Open this footnote Close This problem does not arise out of malice on the part of the AI tool, because it cannot—AI systems do not think and understand in the same way humans do. And because of this cognitive deficit, these systems should never take the place of actual human accountability. Machines have no legal or moral agency, but the confidence with which they’re programmed to respond to our queries, their remarkable ability to mimic human communication, and the relentless marketing from AI companies telling us that human-level intelligence—or even consciousness—is either already here or just around the corner, can lead many to think otherwise. 52Open this footnote Close this footnote 52 See supra note 39 and accompanying text. …Open this footnote Close There are, of course, many practical uses for these technologies, but it would be a grave mistake to turn those specific instances of usefulness into a black box into which we could divert all attempts at human accountability.

Assigning a kind of agency to AI technologies would operate as an accountability sink for law enforcement agencies. Even when examples of AI systems leading police departments to misidentify suspects and make wrongful arrests—often leading to imprisonment for months—are frequent, many law enforcement leaders and government officials either ignore them or see them as temporary problems which will soon be fixed with the next AI tool. Some policing organizations argue that AI automation can actually serve to increase accountability. 53Open this footnote Close this footnote 53 See, e.g., Randy Burkhammer, Priorities in Police Training: Accountability Through Automation, Police Chief Mag., https://perma.cc/‌8D92-A5JP (archived July 24, 2026) (arguing that police use of technologies can be leveraged to increase law enforcement accountability). …Open this footnote Close While this may be true in certain use cases, serious problems, such as sealed knowledge and automation bias described above, 54Open this footnote Close this footnote 54 See supra Parts I, II. …Open this footnote Close will overwhelm most of those benefits while giving the appearance of an efficient, accountable law enforcement organization. As discussed in Part I, Angela Lipps’s arrest by the Fargo Police Department was based entirely on an error generated by their facial recognition system, a technology that remains in general law enforcement use despite the frequency of these kinds of problems. 55Open this footnote Close this footnote 55 Lauren Yu & Nathan Freed Wessler, More Than a Dozen Wrongful Arrests Due to Police Reliance on Facial Recognition Technology, ACLU (Apr. 14, 2026), https://perma.cc/‌QF6S-4GNL. …Open this footnote Close Like most other AI technologies, facial recognition software does not give the user the opportunity to look under the hood to investigate errors, and therefore their black box natures give police departments like Fargo’s the ability to deflect blame and avoid accountability, pointing to their reliance on their technological tools.

Conclusion

Policymakers have struggled with the idea of technology as social fact. 56Open this footnote Close this footnote 56 See generally Ryan Calo, Law and Technology: A Methodical Approach 20-47 (2025) (arguing that technology as a social fact is not an ungovernable or inevitable circumstance, but is the outcome of societal choices and structures). …Open this footnote Close The ways in which we design, deploy, and use the tools we build are inextricably intertwined with the ways in which we interact and create rules and norms around those interactions. The recent advances made by and through AI technologies are indeed remarkable, but considerable care must be taken when we make decisions about their use and suitability for a particular context, especially when those contexts include impacts on civil liberties. AI systems are touted as a means for busy police departments to reduce workloads by doing some of their thinking for them. And we surely must allow room to improve law enforcement’s ability to accomplish their delegated mission in good faith. After all, it would surely be preferable if we could deploy AI technologies in a way that gives police departments an improved ability to investigate or even prevent crime. But technologies are always accompanied by the worldviews of their creators, which may include a reduced respect for due process and police accountability. We should be wary of increasing the list of things we no longer have to think about, as some of those things deserve the benefit of human consideration. Examples of erroneous arrests like that of Angela Lipps will continue to arise due to the inherent problems of AI technologies described in this Essay. Yet law enforcement agencies see these tools as the means to improve outward efficiency by reducing the perceived need for human-led investigations, where evidence and good judgment provide the bases for arrest. Further, as federal agencies have increased their use of AI technologies to support surveillance and arrests, we face a future where unaccountable policing based on flawed, opaque AI technologies becomes the norm. 57Open this footnote Close this footnote 57 Jude Joffe-Block, Immigration Agents Have New Technology to Identify and Track People, NPR (Nov. 8, 2025, 5:00 AM ET), https://perma.cc/‌TMH5-5E9H; Bruce Schneier, Meta is Testing Facial Recognition for Police and Military, Schneier on Sec. (Jun. 26, 2026, 12:40 PM ET), https://perma.cc/‌E8FU-L2BU. …Open this footnote Close Ceding too much cognitive ground to AI tools also means giving up a measure of accountability to those systems—an unacceptable position where policing powers are concerned.

*Associate Professor of Law, Georgia State University College of Law. Thank you to Anthony Kreis, Monica Iyer, Jonathan Todres, and the Stanford Law Review Online editors for their helpful comments.