FOLLOW US

Why Large Language Models Fool Busy Lawyers

Generative AI excels at mimicking legal syntax—Bluebook pin cites, Latin parentheticals, the calm cadence of judicial prose. That linguistic polish triggers “automation bias,” says Northwestern University tech‑ethics professor Daniel Linna: “When the memo matches the house style, senior associates assume the facts line up too.”

Compounding the problem is prompt drift: slight wording changes can shift a model from summarizing real precedent to inventing analogues. Firms that fed their research queries into private GPT instances gained speed but forfeited the guardrails and citation‑checking layers of traditional databases.

Insurers Turn Anxiety into Endorsements

Professional‑liability carriers, alarmed by an uptick in claim notices tied to AI errors, have started offering (and in some cases requiring) “generative‑AI endorsements.” The riders exclude coverage for losses stemming from “unverified or unreviewed” model output. A new survey by Marsh McLennan’s Law‑Firm Risk Group found premiums rose 5% on average for firms that could not demonstrate a written AI‑governance framework.

EPIC Insurance’s annual trends report warns that undisclosed AI use may trigger material‑misrepresentation defenses, letting carriers rescind policies after a malpractice judgment.

Ethics Boards and Courts Tighten the Screws

• Bar Opinions– California, New York, and Florida have issued formal guidance tethering AI use to Model Rule 1.1 (Competence) and 5.1 (Supervisory Duties), insisting on “reasonable testing, verification, and disclosure.”
• Local Rules– The Northern District of Texas and the Eastern District of Missouri now require certificates describing any AI assistance and the human‑verification steps taken. Judges in the Southern District of New York have signaled similar amendments after the Avianca
• Legislative Proposals– A bipartisan House bill would direct the Federal Trade Commission to deem undisclosed AI use in legal services a deceptive trade practice, carrying civil penalties up to $50,000 per violation.

Guardrails in Practice: What Big Law Is Actually Doing

1. Human‑in‑the‑Loop Double Check– Kirkland & Ellis mandates that two licensed attorneys review every AI‑assisted filing, signing an internal attestation stored in matter management software.
2. Prompt‑Engineering CLE– Winston & Strawn’s new eight‑hour “Prompt Craft for Lawyers” course teaches spotting hallucination “tells,” such as volume numbers that exceed a reporter series’ lifespan or citations to unpublished courts in jurisdictions that do not allow them.
3. Chain‑of‑Custody Metadata– Debevoise & Plimpton inserts invisible audit tags logging the exact prompt, timestamp, and model version for each AI‑generated paragraph—providing a forensics trail when questions arise.
4. Red‑Team Simulations– Perkins Coie runs quarterly “hallucination hackathons,” awarding prizes to associates who can trick the firm’s private model into inventing sources, then documenting the fix.

Corporate Clients Push a Hard Bargain

General counsel at Fortune 100 companies, wary of paying to clean up AI mistakes, are rewriting engagement letters: any AI‑induced correction is non‑billable, and repeated offenses trigger fee clawbacks. Insurer State Farm used that clause to recoup discovery costs after the K&L Gates mishap, according to correspondence reviewed by USA Herald. Reason.com