Google's AI Glasses and Implications for Law Enforcement

Google announced a series of partnerships with eyewear companies to develop glasses that incorporate artificial intelligence (AI), marking a significant step in the evolution of wearable technology and its integration into daily life and professional sectors, including law enforcement.

Google's AI Glasses Partnerships

Google has committed up to $150 million to work with Warby Parker on development and sale of AI-powered smart glasses, leveraging the Android XR platform and Gemini AI model[1][2][10]. The initiative extends to partnerships with other eyewear brands, such as Gentle Monster and Kering, and includes a broader collaboration with Samsung to build both the hardware and software foundation for future AR glasses[6][8][10]. The glasses will feature cameras, microphones, and speakers, providing hands-free access to information, live translation, and integration with users’ smartphones[3][10]. Google states that their approach emphasizes making these devices both functional and suitable for all-day wear, with plans to involve developers in building applications for the platform later this year[6][10].

Implications for Law Enforcement

The integration of AI into smart glasses has implications for law enforcement operations:

• Real-Time Data Access and Situational Awareness: AI-enabled glasses can provide officers with immediate access to critical information, such as suspect identification, navigation, and threat assessments, directly within their field of view[4][9][11]. This can streamline investigations, support enforcement actions, and enhance officer safety.

• Facial Recognition and Surveillance: Smart glasses equipped with AI-driven facial recognition can rapidly compare faces in real time against law enforcement databases, aiding in the identification of suspects and missing persons[4][7][11]. Such systems have already been deployed in various jurisdictions, including China, Dubai, and New York, where they have improved the speed and accuracy of suspect recognition[4][7][11].

• Evidence Collection and Communication: The ability to record and transmit evidence in real time, as well as translate languages or communicate with dispatch and other officers, can improve operational efficiency and support community engagement[4][11].

• Privacy and Ethical Concerns: The widespread use of AI-powered smart glasses raises privacy issues. Real-time surveillance and facial recognition capabilities may lead to concerns about data security, potential misidentification, and the erosion of privacy in public spaces[4][5][9]. Research and pilot programs have emphasized the need for ethical frameworks, clear protocols, and legislation to govern the use of such technologies in law enforcement, aiming to balance operational benefits with the protection of civil liberties[4][9][11].

Challenges and Considerations

• Public Trust and Acceptance: The deployment of AI smart glasses by law enforcement requires transparency and public engagement to address concerns about surveillance and misuse[4][9][11].
• Technical and Operational Readiness: Successful integration depends on reliable hardware, effective AI algorithms, and compatibility with existing law enforcement databases and workflows[4][11].
• Legislation and Policy: Policymakers should consider establishing clear guidelines for the appropriate use of smart glasses, including data handling, retention, and oversight mechanisms[4][9].

Conclusion

Google’s partnerships to develop AI-powered smart glasses signal a shift toward more immersive and context-aware wearable technology. For law enforcement, these advancements offer new tools for real-time information access, surveillance, and communication. However, adoption should be accompanied by consideration of privacy, and ethics to ensure lawful use.

Citations:

[1] https://techcrunch.com/2025/05/20/google-commits-150m-to-develop-ai-glasses-with-warby-parker/

[2] https://www.wsj.com/business/warby-parker-partners-with-google-to-develop-intelligent-eyewear-aabedfb2

[3] https://www.cnet.com/tech/computing/exclusive-google-sees-xr-smart-glasses-as-the-ultimate-use-for-ai-with-warby-parker-samsung-and-xreal-on-deck/

[4] https://ijsret.com/2025/03/29/enhancing-law-enforcement-with-smart-glasses-a-comprehensive-investigation-into-real-time-criminal-detection-using-ai-and-machine-learning/

[5] https://www.foxnews.com/tech/facial-recognition-glasses-turn-everyday-life-creepy-privacy-nightmare

[6] https://blog.google/products/android/android-xr-gemini-glasses-headsets/

[7] https://kustomsignals.com/blog/smart-glasses-the-latest-police-weapon-against-crime

[8] https://www.cnbc.com/2025/05/21/sergey-brin-google-glass-android-xr.html

[9] https://pmc.ncbi.nlm.nih.gov/articles/PMC10446065/

[10] https://www.pymnts.com/google/2025/google-to-partner-with-eyewear-brands-on-ai-equipped-smart-glasses/

[11] https://www.police1.com/vision/the-transformative-impact-of-wearable-technologies

[12] https://www.reuters.com/business/google-unveil-ai-upgrades-io-conference-amid-search-challenges-2025-05-20/

[13] https://finance.yahoo.com/news/google-announces-smart-glasses-partnerships-as-it-takes-on-meta-apple-preps-its-own-eyewear-034542364.html

[14] https://www.boozallen.com/insights/velocity/another-set-of-eyes.html

[15] https://www.vuzix.com/blogs/vuzix-blog/smart-glasses-and-the-future-of-law-enforcement

[16] https://www.zdnet.com/article/googles-upcoming-ai-smart-glasses-may-finally-convince-me-to-switch-to-a-pair-full-time/

[17] https://hypebeast.com/2025/5/google-gentle-monster-ai-smart-eyewear-partnership-android-xr-announcement

[18] https://kustomsignals.com/blog/ai-technology-promoting-better-public-safety

[19] https://www.futurepolicing.org/explaining-ai/the-double-edged-sword-of-ai-in-police-reporting

[20] https://www.reddit.com/r/privacy/comments/1khl1xh/meta_has_resumed_research_on_real_time_facial_id/

Understanding False-Positive Hallucinations in AI Research: Implications for Academic Integrity

The emergence of generative artificial intelligence tools transformed academic research. While these tools offer significant assistance in content generation, they also introduce unique challenges-particularly the phenomenon known as "hallucinations." This (AI-assisted) analysis explores the nature, prevalence, and implications of false-positive hallucinations in AI-assisted academic work, along with strategies for educators and students.

Defining False-Positive Hallucinations in AI Research

AI hallucinations occur when large language models (LLMs) generate content that appears factual and authoritative but is actually incorrect, fabricated, or misleading. In the academic context, false-positive hallucinations specifically refer to instances where AI systems confidently present fabricated information as legitimate scholarly content.

Recent research defines hallucinations as occurring "anytime an AI responds incorrectly to a prompt that it should be able to respond correctly to," with outputs presented as facts within an otherwise factual context despite their fundamental flaws[12]. A more nuanced taxonomy categorizes hallucinations based on their degree (mild, moderate, alarming), orientation (factual mirage or silver lining), and specific types such as acronym ambiguity, numeric nuisance, generated golem, virtual voice, geographic erratum, and time wrap[2].

Unlike traditional misinformation, AI hallucinations emerge not from human intent to deceive but from probabilistic processes within the models themselves. This distinction prompted researchers to propose conceptual frameworks that treat AI hallucinations as a distinct form of misinformation requiring specialized understanding and mitigation approaches[10].

The most concerning manifestation in academic contexts is the generation of seemingly legitimate citations and references that don't actually exist. These fabricated references often appear remarkably convincing, using names of authors with previous relevant publications, creating plausible titles, and formatting citations in credible journal styles[3][8].

Types of Academic Hallucinations

Academic hallucinations typically manifest in several forms:

1. Reference fabrication: Generation of non-existent academic sources
2. Fact fabrication: Creation of plausible but false statistical data or research findings
3. Expert fabrication: Attribution of statements to non-existent or misrepresented authorities
4. Methodological fabrication: Description of studies or experiments that never occurred

Prevalence of False-Positive Hallucinations in Academic AI

The prevalence of AI hallucinations in academic contexts is alarmingly high across various disciplines. In the medical domain, a systematic analysis of ChatGPT responses to medical questions revealed that 69% of references provided were completely fabricated, despite appearing authentic and professionally formatted[8]. The responses themselves demonstrated limited quality with a median score of 60% as rated by medical experts, who identified both major and minor factual errors throughout the evaluated content[8].

Legal research tools incorporating AI technologies fare somewhat better but still exhibit significant hallucination rates. An evaluation of leading legal research AI systems including LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) found hallucination rates between 17% and 33%, despite vendors' claims of "hallucination-free" legal citations[14].

These findings contradict optimistic marketing claims that retrieval-augmented generation (RAG) and similar techniques have "eliminated" or "avoided" hallucinations in specialized academic AI tools. While RAG does reduce hallucination rates compared to general-purpose chatbots, it clearly has not resolved the issue entirely[14].

The problem extends beyond specialized tools to general-purpose LLMs used directly by students and researchers. A preliminary investigation of ChatGPT revealed its tendency to generate fake peer-reviewed citations that appear legitimate but are entirely fabricated through predictive processes rather than factual knowledge[3].

Notable Examples of False-Positive Hallucinations in Academia

Several documented instances illustrate the real-world impact of AI hallucinations in academic settings:

Medical Research Hallucinations

A particularly concerning example comes from medical research, where ChatGPT provided responses containing both major and minor factual errors when answering medical questions. In one instance, when prompted to provide references for its claims, the AI generated a citation to a seemingly authoritative journal article-complete with real researcher names and a plausible title-that simply did not exist[8]. This fabrication is especially problematic in medical contexts where treatment decisions might be influenced by such misinformation.

Legal Research Fabrications

In the legal domain, AI research tools have been documented producing non-existent case law citations. A preregistered empirical evaluation demonstrated that proprietary legal AI tools would create citations to court cases that never occurred or substantially misrepresent the holdings of real cases[14]. What makes these hallucinations particularly dangerous is their presentation within otherwise factually accurate content, making detection challenging for even experienced legal professionals.

Academic Writing Distortions

In academic writing contexts, AI tools have generated fake journal articles, conferences, and even entire research institutions that sound plausible but don't actually exist. The fabricated references often follow proper citation formats and include realistic publication dates, journal names, and volume numbers-all contributing to their deceptive authenticity[3].

What makes these examples particularly troubling is the confidence with which AI systems present hallucinated information, often seamlessly integrating fabrications with factual content. This blending makes detection challenging without thorough verification of every claimed source.

Should Instructors Require URLs to Source Materials?

Given the prevalence of AI hallucinations, the question arises whether instructors should require students to provide direct URLs to source materials referenced in their submissions.

Arguments Supporting URL Requirements

Requiring URLs to source materials creates an additional verification layer that could significantly reduce the risk of undetected hallucinations. This approach:

1. Forces students to verify that their sources actually exist before submission
2. Streamlines the instructor's verification process
3. Creates a habit of source verification that serves students throughout their academic careers
4. Aligns with emerging institutional policies on AI use in academic writing[13]

A survey of policies from top universities in English-speaking countries reveals a trend toward requiring transparency in AI use, with explicit guidelines for both academic staff and students[13]. Extending these transparency requirements to include verifiable source links represents a logical evolution of these policies.

Arguments Against Strict URL Requirements

However, mandatory URL requirements present several challenges:

1. Not all legitimate academic sources have accessible URLs (physical books, paywalled content, etc.)
2. The focus on URLs might prioritize digital sources over print resources
3. Implementation could create additional workload for both students and instructors
4. URL verification alone doesn't guarantee content accuracy

The increasing integration of AI writing tools into academic environments suggests that education should focus on responsible use rather than prohibition[5]. Surveys of EFL university students indicate that these tools are already actively used for enhancing writing quality, with tools like Grammarly and ChatGPT being particularly favored[5][11].

Balanced Recommendation

A balanced approach would involve requiring URLs or other definitive source identifiers (DOIs, ISBN numbers, etc.) for key claims and statistics while focusing on broader verification skills. Instructors should develop clear guidelines for AI use in academic writing that emphasize transparency about AI assistance while teaching responsible verification practices[9][13].

Strategies for Students to Address False-Positive Hallucinations

Students can take several proactive steps to minimize the risk of including hallucinated content in their academic submissions:

1. Verify Every Citation

When using AI tools for research assistance or writing support, students should verify every citation and factual claim generated. This verification process should include:

• Confirming the existence of cited sources through library databases or academic search engines
• Cross-checking key facts with multiple reliable sources
• Being particularly cautious with statistical claims and specific numerical data[3][8].
• Use in-text citations associated with the references in a References section of academic submissions. 

2. Use AI as a Supplement, Not a Replacement

Research on EFL students' experiences with ChatGPT suggests that the most successful approach involves using AI as a supporting tool rather than the primary source of content. Students benefit most when using AI for:

• Overcoming uncertainties
• Clarifying vocabulary
• Receiving content suggestions that they then critically evaluate and refine

This approach enhances essay quality by allowing students to focus on creative aspects while maintaining authenticity in their work[11].

3. Develop Critical Evaluation Skills

Students should develop and apply critical thinking skills specifically adapted to evaluating AI-generated content. This includes:

• Questioning implausible or too-perfect statistics
• Being skeptical of convenient but unsourced claims
• Looking for internal consistency in arguments and evidence
• Recognizing that AI tends to provide overly complex suggestions that may lack cultural sensitivity[11]

4. Declare AI Usage Transparently

Transparency regarding AI tool usage is increasingly recognized as an ethical requirement in academic writing. Students should:

• Follow institutional guidelines for declaring AI assistance
• Specify which portions of their work were AI-assisted
• Describe their verification process
• Maintain a clear distinction between AI-suggested content and their original analysis[9]

A study of transparency in academic research journals found that 37.6% of nursing studies journals now require explicit statements about generative AI use in their authors' guidelines, indicating a growing expectation for transparency[9].

5. Utilize Multiple Tools and Approaches

Students can reduce hallucination risks by:

• Using multiple AI tools and comparing outputs
• Employing specialized AI detection tools to identify potentially hallucinated content
• Combining AI assistance with traditional research methods
• Seeking human feedback from peers, writing centers, or instructors[11][13]

Conclusion

False-positive hallucinations represent a significant challenge in AI-assisted academic research, with documented prevalence across disciplines including medicine, law, and general academic writing. The ability of AI systems to generate convincing yet entirely fabricated references and facts threatens academic integrity and the reliability of scholarly work.

Rather than prohibiting AI tools, educational institutions should develop comprehensive policies that balance innovation with integrity. These policies should emphasize transparency, verification, and responsible use while equipping students with the critical skills needed to detect and avoid hallucinated content.

For students, the key to successful navigation of this new landscape lies in developing a balanced approach-leveraging AI's capabilities for improving writing quality and efficiency while maintaining rigorous verification practices and critical evaluation of all AI-generated content.

As AI evolves, so too must the approaches to academic integrity. By acknowledging the challenges posed by hallucinations and implementing targeted strategies to address them, academia can harness the benefits of AI while preserving the values of rigor and honesty.

Citations:

[1] https://www.semanticscholar.org/paper/c7a54be535e0d1f2715b214e598109e05e6f7507

[2] https://arxiv.org/abs/2310.04988

[3] https://www.semanticscholar.org/paper/324bee1adcaafc336f0255980db8544fe883449f

[4] https://arxiv.org/pdf/2401.06796.pdf

[5] https://www.semanticscholar.org/paper/8d716fe6571ef65bf61e4563e58d0d04c190e6ca

[6] https://www.semanticscholar.org/paper/8d3c4171a3f0e873596702688f9318e71bb22d6f

[7] https://arxiv.org/abs/2405.09589

[8] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11975740/

[9] https://pubmed.ncbi.nlm.nih.gov/37904646/

[10] https://arxiv.org/abs/2504.13777

[11] https://www.semanticscholar.org/paper/6656081678c7777d59ada86713fc03beb584caeb

[12] https://www.semanticscholar.org/paper/40804fcbf832bf4642f1329c20cfb3d505d7e7d2

[13] https://www.semanticscholar.org/paper/86dd7119a3f29d66fd379cbcc1241344c8c8e105

[14] https://arxiv.org/abs/2405.20362

[15] https://www.semanticscholar.org/paper/a653040e98acdd2b6bc87f64985b5ace8646de88

[16] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6322788/

[17] https://www.semanticscholar.org/paper/e1fdda1c63b11b42df98a1e2bb348d1853fc6f5f

[18] http://arxiv.org/pdf/2412.15264.pdf

[19] http://arxiv.org/pdf/2405.19648.pdf

[20] https://arxiv.org/pdf/2501.13946.pdf

[21] https://arxiv.org/abs/2406.04175

[22] https://arxiv.org/abs/2410.22349

[23] https://arxiv.org/html/2407.05474v1

[24] https://arxiv.org/html/2502.01812v1

[25] https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence)

[26] https://pmc.ncbi.nlm.nih.gov/articles/PMC10483440/

[27] https://originality.ai/blog/ai-hallucination-factual-error-problems

[28] https://pmc.ncbi.nlm.nih.gov/articles/PMC11681264/

[29] https://www.aryaka.com/blog/ai-hallucinations-in-cybersecurity/

[30] https://www.ibm.com/think/topics/ai-hallucinations

[31] https://www.forbes.com/sites/conormurray/2025/05/06/why-ai-hallucinations-are-worse-than-ever/

[32] https://www.amu.apus.edu/docs/shared/success-center-pdfs/caution-about-ai-hallucinations---all-text.pdf

[33] https://www.datacamp.com/blog/ai-hallucination

[34] https://bhmpc.com/2024/12/ai-hallucination/

[35] https://www.semanticscholar.org/paper/88b6efb0edc940847704c73701f61b82ff85fdf6

[36] https://www.semanticscholar.org/paper/87acf1f6c7f6fcf6bde6b7298063271d10b275d5

[37] http://arxiv.org/pdf/2403.08542v1.pdf

[38] https://arxiv.org/pdf/2406.12221.pdf

[39] https://cloud.google.com/discover/what-are-ai-hallucinations

[40] https://mitsloanedtech.mit.edu/ai/basics/addressing-ai-hallucinations-and-bias/

[41] https://www.techtarget.com/whatis/definition/AI-hallucination

[42] https://hechingerreport.org/proof-points-combat-ai-hallucinations-math/

[43] https://builtin.com/artificial-intelligence/ai-hallucination

[44] https://neilpatel.com/blog/ai-hallucination/

[45] https://lennartnacke.com/how-to-prevent-ai-hallucinations-in-academic-work/

[46] https://arxiv.org/abs/2504.18423

[47] https://www.semanticscholar.org/paper/67e67ac9484d54fd48e5273564271432da5fc398

[48] http://arxiv.org/pdf/2402.19103.pdf

[49] https://www.forbes.com/sites/shashankagarwal/2024/06/12/battling-ai-hallucinations--detect-and-defeat-false-positives/

[50] https://vancouver.northeastern.edu/dont-be-surprised-by-ai-chatbots-creating-fake-citations/

[51] https://www.reddit.com/r/AcademicPsychology/comments/1cbqqv1/false_positive_with_ai_detector/

[52] https://www.nature.com/articles/s41598-023-41032-5

[53] https://www.semanticscholar.org/paper/01b6515d7ccd16d4e46acdb344dbd8f9ca5d7629

[54] https://www.semanticscholar.org/paper/c4f006c02190ddd7ece01542c3f0089c3be6ab7a

[55] https://arxiv.org/html/2411.03108v2

[56] http://arxiv.org/pdf/2406.15130.pdf

[57] https://teche.mq.edu.au/2023/02/why-does-chatgpt-generate-fake-references/

[58] https://www.reddit.com/r/OpenAI/comments/105pib0/chatgpt_gives_me_references_i_cant_find_back/

[59] https://blogs.library.duke.edu/blog/2023/03/09/chatgpt-and-fake-citations/

[60] https://www.enttoday.org/article/chatgpt-generated-fake-references-in-academic-manuscripts-is-a-problem/

[61] https://www.psypost.org/chatgpt-hallucinates-fake-but-plausible-scientific-citations-at-a-staggering-rate-study-finds/

[62] https://guides.library.illinois.edu/generativeAI/hallucinations

[63] https://www.tidio.com/blog/ai-hallucinations/

[64] https://www.asccc.org/sites/default/files/ASCCC_AI_Resources_2024.pdf

[65] https://pmc.ncbi.nlm.nih.gov/articles/PMC10277170/

[66] https://www.semanticscholar.org/paper/7bef4b1b091bd0957e6f2a8eade4053613307ad7

[67] https://arxiv.org/abs/2504.08832

[68] https://arxiv.org/pdf/2401.11817.pdf

[69] https://www.nature.com/articles/s41599-024-03811-x

[70] https://www.coursera.org/articles/ai-hallucinations

[71] https://detecting-ai.com/blog/ultimate-guide-to-hallucination-detection-metrics

[72] https://academic.oup.com/jla/article/16/1/64/7699227

[73] https://www.ecampusnews.com/featured/2023/09/05/detect-ai-generated-text/

[74] https://arxiv.org/html/2401.06796v1

[75] https://pmc.ncbi.nlm.nih.gov/articles/PMC2702442/

[76] https://datascientest.com/en/understanding-ai-hallucinations-causes-and-consequences

[77] https://arxiv.org/abs/2310.10627

[78] https://pubmed.ncbi.nlm.nih.gov/38516933/

[79] https://www.semanticscholar.org/paper/24e9a44f0bf645583603c7ee03bdddd71831bef6

[80] https://www.thesify.ai/blog/teaching-ethical-ai-in-academic-writing-a-guide-for-professors-and-instructors

[81] https://poorvucenter.yale.edu/AIguidance

[82] https://www.montclair.edu/faculty-excellence/ofe-teaching-resources/clear-course-design/practical-responses-to-chat-gpt/red-flags-detecting-ai-writing/

[83] https://www.facultyfocus.com/articles/teaching-with-technology-articles/mitigating-hallucinations-in-llms-for-community-college-classrooms-strategies-to-ensure-reliable-and-trustworthy-ai-powered-learning-tools/

[84] https://community.canvaslms.com/t5/Canvas-Question-Forum/Speedgrader-Website-URL-submission-with-no-preview/td-p/481971

[85] https://www.vpaa.uillinois.edu/digital_risk_management/generative_ai/instructors

[86] https://craigball.net/2023/12/11/policy-for-student-use-of-ai/

[87] https://honorcode.nd.edu/ai-recommendations-for-instructors/

[88] https://www.vanderbilt.edu/generative-ai/academic-integrity/

[89] https://arxiv.org/abs/2504.17550

[90] https://www.semanticscholar.org/paper/04cc443974ce641d3d1de381070d87ff630238db

[91] https://arxiv.org/pdf/2311.05232.pdf

[92] https://library.potsdam.edu/AI_Ethics/Hallucinations

[93] https://www.asapp.com/blog/preventing-hallucinations-in-generative-ai-agent

[94] https://www.mobihealthnews.com/news/study-ai-hallucinations-limit-reliability-foundation-models

[95] https://www.semanticscholar.org/paper/e5f183ffafd74e2fba831420fa1f3e5f07b7ce2d

[96] https://arxiv.org/abs/2402.06647

[97] https://arxiv.org/pdf/2407.08039.pdf

[98] https://arxiv.org/abs/2401.11817

[99] https://www.lakera.ai/blog/guide-to-hallucinations-in-large-language-models

[100] https://www.gigaspaces.com/data-terms/llm-hallucinations

[101] https://pmc.ncbi.nlm.nih.gov/articles/PMC10619792/

[102] https://my.clevelandclinic.org/health/symptoms/23350-hallucinations

[103] https://arxiv.org/abs/2311.05232

[104] https://arxiv.org/html/2504.07863v1

[105] http://arxiv.org/pdf/2403.06710.pdf

[106] https://arxiv.org/pdf/2402.13331.pdf

[107] https://arxiv.org/pdf/2503.00436.pdf

[108] https://arxiv.org/html/2410.10408v1

[109] https://www.semanticscholar.org/paper/a0a26cc12a8dddf91267b21ae703291257cdcf85

[110] https://arxiv.org/abs/2408.09429

[111] https://arxiv.org/abs/2408.07852

[112] https://pubmed.ncbi.nlm.nih.gov/8870325/

[113] https://arxiv.org/abs/2403.17306

[114] https://www.semanticscholar.org/paper/88cdca57a248f7d1d9a0667834dc9f9af24bd2be

[115] https://arxiv.org/pdf/2401.06796.pdf

[116] https://arxiv.org/abs/2503.05777

[117] https://arxiv.org/abs/2401.01572

[118] http://arxiv.org/pdf/2403.12244.pdf

[119] http://arxiv.org/pdf/2502.12187.pdf

[120] http://arxiv.org/pdf/2403.17306.pdf

[121] https://arxiv.org/abs/2308.15126

[122] http://arxiv.org/pdf/2503.06149.pdf

[123] https://arxiv.org/abs/1811.10800

[124] https://www.semanticscholar.org/paper/9ad28ee68c50627ffb107daa0a7d2a48fda270e6

[125] https://www.semanticscholar.org/paper/88bb5edda1ef91b1af2fd22b546c542a6df39ba2

[126] https://www.semanticscholar.org/paper/04a8cf33d0d3ea6b937e791354e62bb3f53d291e

[127] https://www.semanticscholar.org/paper/a851bba1e6f67122eb6a0ad9ae4f55a52d1e3874

[128] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10350445/

[129] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10323419/

[130] https://arxiv.org/abs/2503.14933

[131] https://arxiv.org/pdf/2404.04845.pdf

[132] https://arxiv.org/pdf/2310.06498.pdf

[133] https://arxiv.org/pdf/2306.06085.pdf

[134] https://arxiv.org/pdf/2311.14648.pdf

[135] https://arxiv.org/pdf/2401.03205.pdf

[136] http://arxiv.org/pdf/2503.05806.pdf

[137] https://www.semanticscholar.org/paper/13f004cdf3df3496bcf90e1e07456ea371a484d1

[138] https://www.semanticscholar.org/paper/c9e049592d7630c3805c7b94fd66a7b311547cc6

[139] https://pubmed.ncbi.nlm.nih.gov/37442292/

[140] https://www.semanticscholar.org/paper/c984da7a7689948294693f2f8345f15528b26923

[141] https://arxiv.org/abs/2406.15130

[142] https://www.semanticscholar.org/paper/68bd6debf2edae10c8750d981491589224f0d2e3

[143] http://arxiv.org/pdf/2309.09401.pdf

[144] https://arxiv.org/ftp/arxiv/papers/2308/2308.03301.pdf

[145] http://arxiv.org/pdf/2308.11767.pdf

[146] http://arxiv.org/pdf/2310.05046.pdf

[147] https://arxiv.org/abs/2305.05133

[148] https://arxiv.org/pdf/2402.08467.pdf

[149] http://arxiv.org/pdf/2305.13534.pdf

[150] https://arxiv.org/pdf/2312.11870.pdf

[151] https://www.semanticscholar.org/paper/6bf4e95e63df023c81458ec60a8324788535a2f4

[152] https://arxiv.org/abs/2310.04988

[153] https://www.semanticscholar.org/paper/cf67e6eddedbed026b6e6bebedd010cee8dcc0f5

[154] https://www.semanticscholar.org/paper/c77058caeb3eb79e69b12968d02c11690fbeeb28

[155] https://www.semanticscholar.org/paper/adc549943b8ac320b9d1a2e4a9277ee66de9b048

[156] https://www.semanticscholar.org/paper/1755a4bf7472e9b69a1f53b26b0b86faf315df41

[157] https://www.semanticscholar.org/paper/abb7d581d278ace41100019cb745a1b5c2178c98

[158] https://arxiv.org/pdf/2312.05209.pdf

[159] https://arxiv.org/abs/2405.09589

[160] https://www.semanticscholar.org/paper/69703c222a5c11c2bb7d0a78195f0985e5e5afd0

[161] https://www.semanticscholar.org/paper/b886924472c55d68ddda5bab195c0ecbd145e849

[162] https://www.semanticscholar.org/paper/e99c9b0dd750bece8bef047e4058605c5ab1d79a

[163] https://www.semanticscholar.org/paper/a953654628dee771445ea2da8314adfd24796d55

[164] https://arxiv.org/abs/2311.14720

[165] https://arxiv.org/pdf/2309.06794.pdf

[166] https://arxiv.org/pdf/2501.13946.pdf

[167] https://arxiv.org/pdf/2310.17143.pdf

[168] https://arxiv.org/pdf/2502.05698.pdf

[169] https://arxiv.org/pdf/2304.02478.pdf

[170] https://www.semanticscholar.org/paper/f27dd152582b9da3c3a879025eceaebbf2a50752

[171] https://www.semanticscholar.org/paper/f5c3faa8002fcaa458c023bf6e2f44ec83d89fbe

[172] https://www.semanticscholar.org/paper/1edab696120cd508731a8a20e25bf600e8838a18

[173] https://arxiv.org/abs/2401.06796

[174] https://www.semanticscholar.org/paper/6f1643e182bafe77e12b288b5ffcbfc6110a368f

[175] https://arxiv.org/pdf/2401.06792.pdf

[176] https://arxiv.org/abs/2404.03732

[177] https://arxiv.org/abs/2406.08184

[178] https://arxiv.org/abs/2503.17309

[179] https://arxiv.org/abs/2410.03026

[180] https://arxiv.org/pdf/2402.09733.pdf

[181] https://arxiv.org/html/2404.07461v2

[182] https://arxiv.org/pdf/2402.01769.pdf

[183] https://www.arxiv.org/pdf/2402.06647.pdf

[184] http://arxiv.org/pdf/2502.11113.pdf

[185] https://arxiv.org/pdf/2502.16143.pdf

 

Note: The author was assisted by Artificial Intelligence in the creation of this document. Efforts were made to verify the source material. 

https://kardasz.blogspot.com/2025/05/understanding-false-positive.html

 

Frank Kardasz, May 16, 2025

 

 

AI-Generated Court Filings & the Perils of Fictitious Citations: A U.S. Case Law Overview of Hallucinations

Introduction

The rise of generative AI tools in late 2022 and 2023 (e.g. OpenAI’s ChatGPT) brought new possibilities to the legal profession. Attorneys quickly began experimenting with AI to research and draft court filings. However, these tools are prone to “hallucinations” – a phenomenon where the AI confidently fabricates plausible-sounding but false information, including bogus quotes or case citations (8).

Early on, experts and judges warned that AI chatbots can “make stuff up — even quotes and citations” (4). Despite such warnings, a series of U.S. cases since 2023 revealed attorneys filing AI-generated briefs containing nonexistent case law and fictitious facts, leading to court sanctions and professional embarrassment. This post provides a historical overview of these incidents, examines the legal and ethical ramifications (from court procedure to attorney discipline), and concludes with best-practice recommendations for the responsible use of AI in legal work.

Mata v. Avianca (2023): The First “ChatGPT Lawyer” Sanction

One of the earliest and most widely publicized incidents was Mata v. Avianca, Inc. in New York in 2023 – a cautionary tale that became a wake-up call for the legal community. In this personal injury case against Avianca Airlines, the plaintiff’s attorneys submitted a brief in opposition to a motion to dismiss that cited six seemingly relevant court decisions. The only problem: none of those cases actually existed. They had been invented by ChatGPT, which the lawyer had used for legal research (1).

Opposing counsel and the judge discovered that the cited cases were fictitious when they could not be found in any legal database (1). The court ordered the filing attorney to produce the sources, and in response the attorney filed copies of AI-generated fake opinions purporting to be the missing cases. 

It was only after a further order to show cause that the attorney finally admitted that an AI tool had been used and was the source of the phony citations. By that point, the attorneys had doubled down on the falsehoods for nearly three months, even standing by the “fake opinions” after they were questioned (2).

U.S. District Judge P. Kevin Castel was not amused. In a blistering sanctions order issued in June 2023, Judge Castel found that the attorneys had acted in bad faith, engaging in “acts of conscious avoidance and false and misleading statements to the court” (1). The court emphasized that filing fictitious case law is a serious breach of an attorney’s duties: “Many harms flow from the submission of fake opinions. The opposing party wastes time and money in exposing the deception. The Court’s time is taken from other important endeavors. The client may be deprived of arguments based on authentic judicial precedents. … It promotes cynicism about the legal profession and the American judicial system.”(6). In short, the lawyers violated their duty of candor and wasted judicial resources.

Sanctions were imposed. The two attorneys (and their law firm) were jointly fined $5,000 (1). The judge also ordered them to inform their client of the sanctions and to send letters enclosing the sanctions order to all the judges who had been falsely identified as authors of the fake cases – a humbling form of accountability. In his order, Judge Castel acknowledged that there is nothing “inherently improper” about using AI as a tool for assistance, but he stressed that “lawyer ethics rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings.” (1). In other words, regardless of how an attorney drafts a brief, the attorney bears ultimate responsibility for verifying that the authorities cited are real and valid.

Notably, the lawyers in Mata later claimed they had been unaware that ChatGPT could fabricate case law. In a statement, their firm described the incident as a “good faith mistake”, saying they “fail[ed] to believe that a piece of technology could be making up cases out of whole cloth.” (1). But ignorance was no excuse. The incident swiftly entered legal lore as the “ChatGPT lawyer” fiasco, serving as a stark reminder that novel technology does not absolve attorneys of traditional duties of truthfulness and due diligence.

A Growing Pattern: Subsequent Cases in 2024 and 2025

After the Avianca case, one might have expected attorneys to proceed with caution. Yet, in the following months, similar missteps occurred across the United States, suggesting that the Avianca incident was not an isolated one but rather the beginning of a troubling trend. Judges increasingly found themselves confronting filings tainted by AI-generated falsehoods. By 2025, one federal court remarked that “the epidemic of citing fake cases has continued unabated,” noting several recent examples across jurisdictions (7).

Some notable cases include:

• Gauthier v. Goodyear Tire & Rubber Co. (E.D. Tex. 2024): In a products liability case in the Eastern District of Texas, an attorney filed a brief containing nonexistent case citations apparently supplied by an AI tool. The court sanctioned him with a $2,000 fine and even required the attorney to attend a continuing legal education course on generative AI as part of the penalty (9). The lawyer was also ordered to provide a copy of the sanctions order to his client, underscoring the breach of trust with the client.

• United States v. Hayes (E.D. Cal. 2025): In January 2025, a federal judge in California fined a defense attorney $1,500 for submitting a brief in a criminal case that cited fake case law. The attorney claimed the error was due to an “inadvertent citation error,” but the court found this explanation inadequate. In addition to the fine, the judge directed that a copy of the sanctions order be sent to the state bar, potentially for disciplinary review (10).

• Mid-Central Operating Engineers Fund v. Hoosiervac LLC (S.D. Ind. 2025): A judge in Indiana encountered yet another filing rife with AI-fabricated citations and took an even tougher stance. In February 2025, the court recommended a $15,000 sanction to drive home the seriousness of the misconduct (11). This appears to be one of the heaviest monetary sanctions to date for an AI-related filing violation, reflecting the court’s view that a strong deterrent was needed.

• Wadsworth v. Walmart, Inc. (D. Wyo. 2025): Even large law firms fell prey to AI’s pitfalls. In this products liability case (involving an allegedly defective toy hoverboard), three lawyers from the prominent plaintiffs’ firm Morgan & Morgan filed a set of motions in limine that cited nine non-existent cases (5). The filing attorneys later admitted that they had used an internal AI research tool (a proprietary platform within their firm) which had “hallucinated” these case citations. Once the errors came to light – opposing counsel informed the judge they could not find the cited cases – the lawyers promptly withdrew the faulty filing and candidly acknowledged the mistake to the court. In their submission to the judge, they wrote: “This matter comes with great embarrassment and has prompted discussion and action regarding the training, implementation, and future use of artificial intelligence within our firm.” The Wyoming court imposed sanctions, albeit tailoring them to the attorneys’ respective roles. U.S. District Judge Kelly H. Rankin fined the junior attorney who drafted the AI-driven brief $3,000 and his two supervising co-counsel $1,000 each. Notably, Judge Rankin also revoked the drafting attorney’s pro hac vice admission (his permission to practice in that case as an out-of-state lawyer) as a sanction for the unethical conduct (6). In deciding the sanctions, the judge contrasted the Morgan & Morgan attorneys’ quick candor with the deceit in Mata, observing that unlike in the Avianca matter – where the lawyers misled the court for months – the Walmart case lawyers had been “forthcoming, honest, and apologetic” once the issue was identified (6). The court credited their remedial steps (they had already paid the opposing side’s attorney fees related to the motion and instituted firm-wide AI training and safeguards but still found that a penalty was necessary given the lapse in due diligence (8).

• Other Recent Examples: In Dehghani v. Castro (D.N.M. Apr. 2025), the court imposed sanctions after counsel submitted “non-existent cases” that were “likely the handiwork of a ChatGPT or similar ... AI program’s hallucinations” (12).

• And in April 2025, a federal magistrate judge in the Eastern District of New York sanctioned an attorney for using an AI tool called “ChatOn” to generate fake case citations in a motion to remand (the Guerline Benjamin v. Costco Wholesale Corp. case) (7). That judge opened his opinion by lamenting that these “phony submissions” are causing real problems: an “attorney who submits fake cases clearly has not read those nonexistent cases, which is a violation of Rule 11… These made-up cases create unnecessary work for courts and opposing attorneys alike. And perhaps most critically, they demonstrate a failure to provide competent representation to the client.”. Such blunt language underscores that by 2025, courts viewed this pattern as a national problem requiring a firm response.

In sum, since the watershed Avianca incident in early 2023, there has been a streamof cases in which U.S. lawyers faced sanctions for filing AI-generated falsehoods. The sanctions have varied in severity – from monetary fines to non-monetary measures like compulsory CLE training on ethics and AI , letters of apology or notification to affected parties, referrals to disciplinary authorities, and even temporary or permanent practice restrictions in the case at hand. But across all these cases, the message from the judiciary has been consistent: lawyers will be held accountable for inaccuracies in their filings, regardless of whether those mistakes stem from an algorithm or a human junior associate.

Ramifications: Procedural, Ethical, & Reputational Consequences

These cases have serious ramifications for court procedure, legal ethics, and attorney credibility. Fundamentally, an attorney submitting AI-fabricated information to a court runs afoul of core professional obligations and undermines the integrity of the judicial process.

• Disruption of Court Processes: When a filing contains nonexistent authority or false information, it forces the court and opposing counsel to spend time and resources unraveling the fiction. Judges have had to issue orders to show cause, hold hearings, and write lengthy sanctions opinions – time that could have been devoted to real cases. As one court put it, such antics “create unnecessary work for courts and opposing attorneys alike” (7). In Mata v. Avianca, for example, the case was sidetracked for months by the need to investigate and address the fake citations, including a special hearing where the lawyers were questioned under oath (1). The distraction can also delay resolution of the client’s claims. In the Walmart case, the plaintiffs had to withdraw their motions in limine entirely, effectively losing the opportunity to have legitimate arguments heard at trial on those points (6). Fake citations waste judicial time, drive up litigation costs, and knock proceedings off-course.

• Violations of Ethical Duties: An attorney who files a document with false citations or facts likely breaches multiple ethics rules. The duty of candor to the tribunal (ABA Model Rule 3.3) is paramount – lawyers must not knowingly make false statements of fact or law to a court. In these AI cases, the lawyers at minimum failed to confirm whether what they were submitting was true, and in some instances, they persisted in asserting the truth of fake authorities even after doubts were raised. The court in Mata explicitly found the attorneys made “false and misleading statements to the court,” violating their candor obligation. Even if the lawyers did not set out to lie, Federal Rule of Civil Procedure 11 imposes a duty to make a reasonable inquiry into the factual and legal basis of any filing. Citing opinions that one has never actually read (because they don’t exist) is a clear violation of Rule 11’s requirement that legal contentions be warranted by existing law or a nonfrivolous argument for its extension. In the Costco case, the judge noted that an attorney citing fake cases “clearly has not read those nonexistent cases,” failing the Rule 11 obligation. Many courts found these violations sanctionable even absent a finding of subjective bad faith, since an objective lapse in due diligence is enough for Rule 11 sanctions.

• Competence and Diligence: In addition to candor, the duty of competence and diligence (ABA Model Rules 1.1 and 1.3) is at stake. Lawyers must perform legal research and writing with the thoroughness and preparation reasonably necessary for the representation. Relying on unverified AI output betrays a lack of competence in using technology and in legal research methodology. As one commentator observed, such conduct likely violates “their state’s corollary to the ABA Model Rules of Professional Conduct 1.1 (Competence) and 1.3 (Diligence)” (3). It also implicates Model Rule 3.3’s duty of candor, since the lawyer is effectively misrepresenting fake cases as if they were real. In short, a lawyer cannot blame the machine – using an AI tool does not absolve the lawyer from knowing what is in their filings. The ethical responsibility remains squarely with the human lawyer to ensure accuracy.

• Sanctions and Disciplinary Action: The immediate consequence in these cases has been court-imposed sanctions under Rule 11 or the court’s inherent powers. As detailed above, sanctions have included fines, orders to complete training or community service, and requirements to notify clients, other judges, or disciplinary bodies of the misconduct. Monetary sanctions, while sometimes modest in amount, send a strong signal and become part of the attorney’s record. Some judges have explicitly referred offending lawyers for potential disciplinary review by state bar authorities. In one case, the appellate court even referred an attorney to a grievance committee for possible suspension after a pattern of filing fictitious citations came to light. Beyond formal sanctions, being the subject of a public sanctions order (often reported in legal media and visible on databases) is itself damaging. The attorneys in Mata v. Avianca and Wadsworth v. Walmart found themselves mentioned in nationwide news coverage and became cautionary examples in law offices and CLE programs. Such notoriety can haunt a lawyer’s reputation for years.

• Loss of Credibility and Client Trust: An attorney’s credibility before the court is important – once tarnished, it is hard to restore. Filing a brief full of lies (even unintentional ones) is a fast way to lose a judge’s trust. Judges may become skeptical of that lawyer’s future filings, subjecting them to greater scrutiny or doubt. In the Avianca matter, for instance, the judge’s requirement that the lawyers send copies of the sanctions order to all the judges falsely cited as authors of the fake cases was not only punitive but also effectively flagged those attorneys’ names to multiple jurists. Likewise, the Walmart case attorneys had their right to appear pro hac vice curtailed – a clear message that they had lost the court’s confidence. There is also the damage to the attorney-client relationship. Clients expect and deserve diligent, competent representation. If a lawyer’s misuse of AI leads to embarrassment or setback in a case (as happened when motions had to be withdrawn, or cases got delayed), the client’s interests are harmed. In Mata, the client’s case was ultimately dismissed on procedural grounds, but not before his lawyers became mired in the sanctions dispute. In Wadsworth, the clients effectively forfeited certain pre-trial motions due to their lawyers’ mistake. At a minimum, the client is put in the uncomfortable position of receiving a court-mandated letter confessing their lawyer’s missteps. All of this erodes trust. As one federal judge noted, misuse of AI by lawyers “promotes cynicism about the legal profession and the American judicial system”. This cynicism can extend to clients, who may question whether their lawyers are exercising sound judgment or simply relying on dubious shortcuts.

• Broader Impacts on the Profession: The legal community has taken note of these AI-related blunders. Bar associations and ethics committees are now actively discussing the proper use of AI. The American Bar Association has cautioned that while AI can be a useful aid, lawyers remain ultimately responsible for the accuracy of their work. Several state and local bar groups have issued ethics opinions reminding attorneys that using AI tools does not diminish obligations under the Rules of Professional Conduct, such as competence, confidentiality, and candor. Moreover, judges around the country have started instituting new procedural safeguards. For example, in the wake of the Avianca episode, a federal judge in Texas (Judge Brantley Starr of N.D. Texas) issued a standing order requiring that any attorney appearing in his court must file a certificate attesting either that no portion of a filing was drafted by generative AI, or that any AI-generated content was verified for accuracy by a human being (4). The order bluntly warns that AI output is not reliable: “These platforms in their current states are prone to hallucinations and bias. On hallucinations, they make stuff up — even quotes and citations.” Other courts have adopted similar rules or guidelines requiring disclosure of AI use in filings. Law firms, too, have reacted by developing internal policies for the use of AI in legal research and writing. After the sanctions in the Walmart case, Morgan & Morgan’s attorneys affirmed that the incident prompted new training and safeguards on AI use firm-wide. In short, the profession is adjusting: there is growing recognition that attorneys must treat AI tools with caution and must train themselves and their teams on how to use these tools responsibly.

Best Practices for Responsible Use of AI in Legal Work

In light of the above, what lessons can legal professionals draw to avoid the pitfalls of AI-generated falsehoods? Below are several best-practice recommendations for attorneys using AI tools in any aspect of legal drafting or research:

1. Verify Every Citation and Quote: “One thing remains the same – checking and verifying the source.” This timeless mandate, as Judge Rankin put it, “remains unchanged” even when using AI. Never cite a case, statute, or quote that you have not personally checked in a trusted primary source. If an AI tool suggests a case or legal principle, find the case via Westlaw, Lexis, or official reporters and read it in full to confirm it actually exists and stands for the proposition asserted. Under no circumstances should an attorney copy-paste output from ChatGPT (or any AI) into a brief without rigorous vetting. The lawyer’s role as a gatekeeper of accuracy is non-delegable.

2. Understand AI’s Limitations – Don’t Trust, Verify: Treat generative AI as an unreliable first draft assistant, not an authoritative source. These tools will at times “hallucinate” – e.g., fabricate case law, fake quotes, or misstate facts. Always assume the possibility that the AI’s output may be wrong or even invented. Approach its suggestions with healthy skepticism. If the AI cites a case that sounds unfamiliar or on-point, be especially suspicious and double-check it. Remember that AI has no accountability or understanding of truth: “Unbound by any sense of duty, honor, or justice, [AI] programs act according to computer code rather than conviction,” as one court observed. The burden is on you to verify everything.

3. Use AI as a Supplement – Not a Substitute – for Traditional Research: AI tools might be helpful for brainstorming search terms, summarizing known documents, or generating preliminary drafts. But they should not replace standard legal research through reliable databases or reading of actual source materials. For finding law, there is no shortcut to using verified research tools and reading the applicable authorities. AI can sometimes point you toward a line of cases, but treat that only as a lead, not an answer. Always cross-reference AI research results with real databases. As Judge Starr warned, lawyers “can’t just trust those [AI] databases. They’ve got to actually verify it… through a traditional database”(4). In practice, this means if ChatGPT cites Smith v. Jones (Imaginary 5th Cir. 2010), you must attempt to locate Smith v. Jones in Westlaw or another source. If you can’t find it readily, that’s a red flag that it likely doesn’t exist.

 4. Maintain Transparency and Candor: If you do use AI in preparing a filing, consider being transparent (to the extent required or appropriate) about your process. Some courts now require attorneys to certify that they have checked their AI-derived content. Even if not required, it may be wise to internally document how you verified any AI-generated research. Never conceal a mistake or double down on a falsehood – that will only compound the ethical breach. If an error is discovered in a filed document (AI-related or not), immediately disclose it and correct the record with the court. Trying to cover it up (as happened initially in Mata) can lead to far worse consequences. Candor in remediation can mitigate sanctions, as seen in the Walmart case where the judge noted the attorneys’ honesty and remedial efforts in deciding on lesser sanctions.

 5. Preserve Client Confidentiality: Be mindful that using cloud-based AI tools could risk exposing confidential or privileged information. Most public AI chatbots retain input data. Avoid inputting any sensitive client facts or case specifics into a public AI tool. If your firm utilizes a proprietary or in-house AI platform (as some larger firms are developing), ensure it has proper data security and that its use is compliant with confidentiality rules. Always obtain client consent if there is any possibility that client data might be shared with an AI service outside the firm. Responsible AI use includes protecting client secrets as required by Model Rule 1.6.

6. Stay Educated on Evolving Standards: The landscape of AI in law is rapidly changing. Courts and bar associations continue to issue new guidelines, ethics opinions, and standing orders regarding AI. Attorneys should keep abreast of these developments – for example, by reading ethics committee reports, attending CLEs on AI in practice, and reviewing any standing orders of judges before whom they appear. Being ignorant of a court’s AI-related requirement (like Judge Starr’s certification rule) could itself lead to sanctions or at least judicial ire. In addition, competence today includes technological competence: comment 8 to Model Rule 1.1 advises lawyers to stay familiar with the “benefits and risks” of relevant technology. Make it a point to understand how generative AI works and its failure modes. The more you know about the tool, the better you can use it wisely (or decide not to use it in a given task).

 7. When in Doubt, Do It the Traditional Way: Finally, if you are unsure about the reliability of an AI output or you cannot practically verify an AI-derived assertion, err on the side of caution. It is far better to spend extra time doing conventional research than to risk your professional reputation and client’s case on an unverified AI result. Use AI’s efficiencies in low-stakes drafting (like generating form language or summarizing non-legal text) rather than in tasks that require absolute accuracy (like citing legal precedents). Exercising prudent judgment about when AI is or isn’t appropriate is now part of the attorney’s skillset.

Conclusion

The growth of AI tools in the legal field has been met with both excitement and trepidation. The U.S. cases since 2023 involving AI-generated fictitious citations serve as lessons in the importance of attorney oversight and integrity. Judges have made clear that while using AI in law practice is not forbidden, doing so irresponsibly – without the requisite human diligence – can lead to real penalties, from fines and sanctions to damaged careers. In the words of the court in the Avianca case, “existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings”(1). No matter how advanced our tools become, the lawyer’s fundamental duties of candor, competence, and diligence remain unchanged.

Legal professionals can harness the power of generative AI to enhance efficiency and productivity. But they must do so wisely – verifying all information, double-checking sources, and never relinquishing their critical oversight. The reputation of attorneys and the integrity of the judicial process depend on it. By learning from the early cautionary cases and adhering to best practices, lawyers can avoid the pitfalls of “hallucinated” legal filings and instead ensure that technology augments, rather than undermines, the quality of their advocacy.

Sources:

1. Sara Merken, “New York lawyers sanctioned for using fake ChatGPT cases in legal brief,” Reuters (June 22, 2023). https://www.reuters.com/legal/new-york-lawyers-sanctioned-using-fake-chatgpt-cases-legal-brief-2023-06-22/#:~:text=The%20judge%20found%20the%20lawyers,misleading%20statements%20to%20the%20court
 
2. Seyfarth Shaw LLP, “Update on the ChatGPT Case: Counsel Who Submitted Fake Cases Are Sanctioned,” Legal Update (June 26, 2023). https://www.seyfarth.com/news-insights/update-on-the-chatgpt-case-counsel-who-submitted-fake-cases-are-sanctioned.html#:~:text=York%20case%20involving%20lawyers%20who,nothing%20to%20do%20with%20AI
 
3. Onika K. Williams, “Use of ChatGPT for Research Leads to Bogus Cases, Sanctions,” ABA Litigation News (Nov. 1, 2023). https://www.americanbar.org/groups/litigation/resources/litigation-news/2023/use-chatgpt-research-bogus-cases-sanctions/#:~:text=Summary
 
4. Jacqueline Thomsen, “US judge orders lawyers to sign AI pledge, warning chatbots ‘make stuff up’,” Reuters (June 2, 2023). https://www.reuters.com/legal/transactional/us-judge-orders-lawyers-sign-ai-pledge-warning-they-make-stuff-up-2023-05-31/#:~:text=In%20an%20interview%20Wednesday%2C%20Starr,information%20without%20verifying%20it%20themselves
 
5. Sara Merken, “Lawyers in Walmart lawsuit admit AI ‘hallucinated’ case citations,” Reuters (Feb. 10, 2025). https://www.reuters.com/legal/legalindustry/lawyers-walmart-lawsuit-admit-ai-hallucinated-case-citations-2025-02-10/#:~:text=
 
6. Wadsworth v. Walmart Inc., No. 23-CV-118, 2025 WL 608073 (D. Wyo. Feb. 24, 2025) (Order on Sanctions). https://caselaw.findlaw.com/court/us-dis-crt-d-wyo/117003959.html#:~:text=Many%20harms%20flow%20from%20the,claiming%20doubt%20about%20its%20authenticity 
 
7. Guerline Benjamin v. Costco Wholesale Corp., No. 24-CV-6213, 2025 WL 2403482 (E.D.N.Y. Apr. 19, 2025) (Sanctions Order). https://caselaw.findlaw.com/court/us-dis-crt-ed-new-yor/117206693.html#:~:text=course%20of%20litigation%2C%20Plaintiff%20has,1
 
8. Eugene Volokh, “Sanctions on Lawyers for Filing Motion Containing AI-Hallucinated Cases,” The Volokh Conspiracy (Feb. 25, 2025). https://reason.com/volokh/2025/02/25/sanctions-on-lawyers-for-filing-motion-containing-ai-hallucinated-cases/#:~:text=,attorneys%20had%20to%20manually%20cross

 

9. JAMES GAUTHIER, Plaintiff, v.GOODYEAR TIRE & RUBBER CO., Defendant.

Civil Action No. 1:23-CV-281. United States District Court, E.D. Texas. November 25, 2024. https://scholar.google.com/scholar_case?case=12295395131698828317&hl=en&as_sdt=2006

 

10. UNITED STATES v. DARAGH FINBAR HAYES (2025). United States District Court, E.D. California. UNITED STATES, Plaintiff, v. DARAGH FINBAR HAYES, Defendant.. Case No. 2:24-cr-0280-DJC. Decided: January 17, 2025. https://caselaw.findlaw.com/court/us-dis-crt-e-d-cal/116862866.html

11. MID CENTRAL OPERATING ENGINEERS HEALTH AND WELFARE FUND, Plaintiff, HOOSIERVAC LLC, Defendant. No. 2:24-cv-00326-JPH-MJD. United States District Court, S.D. Indiana, Terre Haute Division. February 21, 2025. https://scholar.google.com/scholar_case?case=1143802252603162019&q=Mid-Central+Operating+Engineers+Fund+v.+Hoosiervac+LLC+(S.D.+Ind.+2025)&hl=en&as_sdt=2006

12. AZADEH DEHGHANI, Petitioner, v. No. 2:25-cv-0052 MIS-DLMDORA CASTRO, Otero Processing Center Warden. https://law.justia.com/cases/federal/district-courts/new-mexico/nmdce/2:2025cv00052/511942/28/