Every lawyer has heard some version of this story by now, but it’s worth retelling because the details are the lesson.
In 2023, a federal judge in the Southern District of New York sanctioned two lawyers who filed a brief citing six fictitious cases, all produced by ChatGPT. The court found subjective bad faith and imposed $5,000 in sanctions.
But the judge, Kevin Castel, also said something firms tend to forget in the retelling: “There is nothing inherently improper about using a reliable artificial intelligence tool for assistance.” The problem in Mata v. Avianca was never that the lawyers used AI. It’s that none of them verified what it produced before it went to court.
Rule 11 and its state equivalents require you as the lawyer to be the gatekeeper for any work coming out of your firm. You cannot delegate that duty to a chatbot, and after Mata, no judge will accept “the AI told me” as an excuse. The duty to supervise remains a duty and extends to any work produced by an AI tool.
Build a Citation Integrity Protocol
The fix is not complicated, but it has to be non-negotiable. Five rules will keep your firm out of the next sanctions order:
- Ban citing any case not personally pulled from an authoritative source — the court’s own website, Westlaw, Lexis, or Google Scholar with the full opinion reviewed.
- Require a second-person citation audit on every brief before it’s filed. The drafting attorney should never be the sole check.
- Treat every AI-generated citation as “unverified” until someone confirms it against the source.
- Never ask AI to “find cases holding X” without a mandatory verification step attached to the request.
- Document the protocol in your firm’s AI policy — it demonstrates good faith if anything ever does slip through.
Think of it as a route from draft to court-ready, with one hard rule at every stage: AI generates a draft with issues flagged, the attorney identifies which propositions need authority, every case gets pulled directly from Westlaw, Lexis, or the court site, a second person audits the citations, and only then does the attorney sign and file. Any citation that tries to skip the “pull from source” step is, by definition, off route.
Prompts That Reduce Risk
The way you prompt AI tools matters as much as the review step that follows. Three examples worth building into your firm’s templates:
Legal research (issues only): “Identify the legal issues and search terms for [brief type] under [jurisdiction] based on these facts: [facts]. Do NOT cite cases, statutes, or rules. I will verify all authorities in [Westlaw/Lexis].”
Drafting (outline only): “Draft an outline for a [motion type]. Flag every proposition requiring a legal citation. Do not invent citations or quotations.”
Pre-filing review: “Review the attached draft. List every factual assertion and legal conclusion that requires a record cite or controlling authority. List any citation that appears incomplete or inconsistent with standard format.”
None of these prompts ask AI to generate law — they ask it to help you find where verification is needed, which is a fundamentally safer use of the tool.
Choose the Right Tool for the Job
Use authenticated legal-research AI for questions of authority, not general-purpose chatbots. Use document-grounded analysis for facts that are already in the record, not for generating new precedent. And where you have a choice of platform, prefer ones that link citations directly back to source documents, so verification is a click instead of a separate research project.
Every AI citation is “unverified” until pulled from source. Using AI is fine. Filing fake cites is not.
Assume Your AI Chats Are Discoverable
“Private chat” is a dangerous assumption once litigation starts. Courts are actively deciding whether AI conversations are protected by attorney-client privilege or work-product doctrine — or whether they’re discoverable like any other evidence, and the authority is genuinely split.
In United States v. Heppner (S.D.N.Y.), Judge Rakoff treated communications with consumer-grade AI as akin to speaking in a “crowded public elevator” — not protected. In Warner v. Gilbarco (E.D. Mich.), a different court reasoned that AI could be treated as a tool rather than a third-party confidant, depending on the facts. Until that split resolves, the practical rule is straightforward: assume prompts and outputs may be requested in discovery unless your ethics counsel has approved a specific enterprise architecture and privilege strategy for your firm.
What will opposing counsel actually go looking for? Prompts containing facts, damages figures, strategy, or admissions. Iterative chats that show how your theory of the case evolved over time. Exports from ChatGPT, Copilot, Claude, or firm-embedded assistants. And increasingly, client use of AI without counsel’s guidance — a growing and under-appreciated exposure point.
Three firm-level actions close most of that gap:
- Add AI tools to your litigation hold and document retention policies, the same way you would any other communication channel.
- Instruct clients directly: do not use consumer AI to discuss case strategy without your protocol in place.
- Use enterprise instances with contractual confidentiality protections wherever possible, rather than free or consumer tiers.
If your matters involve call recording or transcription AI in New York, NYC Bar Formal Opinion 2025-6 requires obtaining client consent to record and verifying accuracy before you rely on the output — a good model for any jurisdiction, whether or not it’s technically required where you sit.
Verification Is the New Standard of Care
Mata v. Avianca is nearly two years old, and it’s no longer news — which is exactly why it’s dangerous. Familiarity breeds complacency, and complacency is how the next sanctions order gets written. As ABA Formal Opinion 512 puts it, a lawyer must review the work product of generative AI just as the lawyer would review the work of a nonlawyer staff member or contract lawyer. Build the protocol once, apply it every time, and treat every AI citation as unverified until you’ve personally confirmed otherwise.
Want the full citation-integrity protocol, the discoverability case law, and a firm-wide AI governance checklist in one place? Download LAWCLERK’s complete whitepaper, How AI Is Reshaping the Attorney Role.
This article was developed with the assistance of AI tools and edited by Kristin Tyler.
FAQ: AI Citation Risk, Verification, and Discovery
What happened in Mata v. Avianca, and why does every firm still talk about it?
In 2023, a federal judge in the Southern District of New York sanctioned two lawyers $5,000 for filing a brief that cited six fictitious cases generated by ChatGPT. Judge Kevin Castel was clear that using AI wasn’t the problem — the failure was that nobody verified the citations before filing. It remains the standard cautionary tale because the same gap — AI output going straight into a filing without a source check — is just as easy to repeat today.
Is it improper for lawyers to use AI tools like ChatGPT for legal work?
No. Judge Castel himself said “there is nothing inherently improper about using a reliable artificial intelligence tool for assistance.” Rule 11 and its state equivalents make the attorney the gatekeeper regardless of what tool produced a draft, so using AI is fine — filing unverified AI output is what creates liability.
What is a citation integrity protocol?
It’s a firm-wide, non-negotiable set of steps that treats every AI-generated citation as unverified until someone personally pulls it from an authoritative source — the court’s own site, Westlaw, Lexis, or Google Scholar. A complete protocol includes a second-person audit on every brief, a ban on citing anything not personally confirmed, and documentation in the firm’s AI policy so it demonstrates good faith if something ever does slip through.
How should attorneys prompt AI tools to reduce hallucination risk?
Ask AI to identify legal issues, flag propositions that need authority, or list assertions requiring a record cite — never to generate or supply the citations themselves. A safe research prompt explicitly instructs the tool not to cite cases, statutes, or rules and states that the attorney will verify all authorities directly in Westlaw or Lexis, which shifts AI from generating law to flagging where verification is needed.
Are conversations with AI tools protected by attorney-client privilege?
It’s unsettled, and the case law is split. In United States v. Heppner, Judge Rakoff compared consumer AI chats to speaking in a “crowded public elevator” — not protected — while Warner v. Gilbarco reasoned AI could be treated as a tool rather than a third-party confidant depending on the facts. Until that split resolves, firms should assume prompts and outputs may be discoverable unless ethics counsel has approved a specific enterprise architecture and privilege strategy.
Should AI chats be included in litigation holds?
Yes. AI tools should be added to litigation hold and document retention policies the same way any other communication channel would be, since opposing counsel may seek prompts containing facts, damages figures, strategy, or admissions, along with iterative chats showing how a case theory developed.
What does ABA Formal Opinion 512 require when it comes to reviewing AI work product?
It requires a lawyer to review the work product of generative AI with the same scrutiny the lawyer would apply to work from a nonlawyer staff member or contract lawyer. That means verification isn’t optional or occasional — it’s the standard of care every time AI touches a filing.
What’s the single rule that prevents most AI citation problems?
Treat every AI-generated citation as “unverified” until it has been personally pulled from source. Pair that with a second-person audit before filing, and the two safeguards close the gap that produced the sanctions in Mata v. Avianca.