Inside This Report

• California’s SB 574 would prohibit lawyers from delegating legal judgment to generative AI.
• Attorneys would have to personally verify citations and disclose AI use in court filings.
• The bill now awaits Governor Gavin Newsom’s signature or veto.

By Samuel López | USA Herald

California lawmakers have unanimously approved a sweeping bill that would place enforceable guardrails around lawyers’ use of generative artificial intelligence, confronting a growing threat to the integrity of court proceedings—AI-generated cases, citations and legal claims that do not exist.

Senate Bill 574, authored by Democratic state Sen. Tom Umberg, cleared the Assembly on August 31 by a 75-0 vote. The Senate then unanimously approved the Assembly amendments, sending the measure to Gov. Gavin Newsom.

If signed, the legislation would establish California as the first state to enact comprehensive statutory rules governing how attorneys and arbitrators may use generative AI in legal proceedings.

The bill delivers a direct warning to the legal profession. An attorney “shall not delegate the practice of law” to generative artificial intelligence.

That language draws a critical boundary between using AI as an analytical tool and allowing software to perform the independent professional judgment entrusted to a licensed attorney.

Lawyers Would Remain Fully Responsible

Under SB 574, attorneys using generative AI would be required to take reasonable steps to verify the accuracy of its output, including every case and statutory citation. Lawyers must also correct erroneous or fabricated material before using it.

The measure goes further by amending California’s Code of Civil Procedure to prohibit court filings from containing citations that the responsible attorney has not personally verified—even when the citation was supplied by an associate, staff member or AI platform.

That provision could give judges a more explicit basis for imposing sanctions when attorneys submit fictitious authorities or other unverified material.

The legislation responds to a growing series of cases in which lawyers nationwide have faced sanctions after AI systems generated convincing—but nonexistent—judicial decisions. These fabricated results, commonly called “hallucinations,” can include fake quotations, invented case names and citations pointing to unrelated rulings.

Umberg said the integrity of the judicial system depends on courts being able to trust litigants and their advocates. Blaming artificial intelligence for false information, he argued, cannot excuse a lawyer’s failure to verify the work.

Disclosure and Confidentiality Rules

SB 574 would require attorneys to disclose their use of generative AI to the court for every document submitted to it. Lawyers would also have to consider disclosure when AI is used to create content presented to the public.

The bill separately protects sensitive information. Attorneys could not enter confidential, personally identifying or other nonpublic material into an AI system unless access to that information is restricted to the attorney and other authorized individuals who are obligated to preserve its confidentiality.

Protected information includes Social Security and driver’s license numbers, medical or psychiatric records, financial data, account numbers and the addresses or phone numbers of parties, witnesses, victims and court personnel.

AI Cannot Decide Arbitration Cases

The legislation also reaches arbitration. Arbitrators could use AI for assistance, but they could not delegate any part of their decision-making authority to it.

An arbitrator relying on AI-generated information from outside the evidentiary record would have to disclose that use to the parties and, where practical, allow them an opportunity to respond.

SB 574 builds upon existing duties of competence, diligence, confidentiality and candor already imposed on California attorneys. Critics have consequently characterized portions of the measure as duplicative.

Its significance, however, lies in transforming developing ethical expectations into express statutory commands. The message from Sacramento is unmistakable: AI may assist the lawyer, but it cannot replace the lawyer—and responsibility for every word filed in court remains human.

About the Author

Samuel López is a Senior Legal Analyst, investigative journalist and legal researcher with more than two decades of experience examining litigation, court filings and complex legal controversies. His background in civil litigation, legal research and advanced motion practice gives him a practitioner-level understanding of attorney accountability, evidentiary integrity and the emerging risks created by artificial intelligence in the justice system.

Continue Following the Cases That Matter

Subscribe to the USA Herald newsletter and stay ahead of the stories shaping law, government and public accountability.

👉 Follow USA Herald on X at @RealUSAHerald