The Great Bar Exam Debacle
Problems with the February 2025 bar exam were apparent from the start. Hundreds of applicants reported technical meltdowns: crashes before tests even began, screen freezes, disappearing essays, and devastating lags that prevented completion.
But what wasn’t immediately known is what made this exam different. In a shocking admission this week, the State Bar revealed that 23 of the scored multiple-choice questions were created by ACS Ventures, its hired psychometrician, using artificial intelligence.
Leah Wilson, the State Bar’s executive director, issued a statement attempting to contain the fallout:
“We have confidence in the validity of the [multiple-choice questions] to accurately and fairly assess the legal competence of test-takers.”
Yet confidence in the Bar itself is rapidly eroding. Mary Basick, assistant dean at UC Irvine Law School said,
“The debacle that was the February 2025 bar exam is worse than we imagined,” Mary Basick, assistant dean of academic skills at the UC Irvine Law School said. “I’m almost speechless. Having the questions drafted by non-lawyers using artificial intelligence is just unbelievable.”
Katie Moran, a professor at the University of San Francisco School of Law, echoed the alarm: “It’s a staggering admission.”
“The State Bar has admitted they employed a company to have a non-lawyer use A.I. to draft questions that were given on the actual bar exam,” she said. “They then paid that same company to assess and ultimately approve of the questions on the exam, including the questions the company authored.”
In essence, the gatekeeping institution tasked with ensuring lawyers are qualified is now being accused of lowering the standards—by turning to outsourced non-lawyer A.I. developers to write the very tests that license the profession.
It gets worse: according to the State Bar’s own admission, Kaplan drafted 100 of the 171 scored multiple-choice questions, and 48 more were recycled from a first-year law students’ exam. Only a minority came directly from traditional lawyer-drafted sources.
