FOLLOW US

A groundbreaking Harvard study is sending shockwaves through the medical world after finding that artificial intelligence systems outperformed human doctors in emergency triage diagnoses, a critical moment when patients first arrive at the hospital and decisions can mean life or death.

The findings, described by researchers as a potential turning point in healthcare, suggest AI may already be reshaping how medicine is practiced in high-pressure environments.

A Turning Point in Emergency Medicine

From Hollywood portrayals like George Clooney in ER to Noah Wyle in The Pitt, emergency room doctors have long been seen as indispensable heroes. But new research suggests the balance of expertise in the ER may be shifting in unexpected ways.

The study, conducted by Harvard researchers and published in the journal Science, tested hundreds of doctors against advanced AI systems in simulated emergency scenarios. The results showed large language models performed more accurately in diagnosing patients during early triage.

The authors said the results showed large language models “have eclipsed most benchmarks of clinical reasoning”.

Independent experts called the findings “a genuine step forward” in artificial intelligence applied to medicine.

AI vs Doctors in Real Emergency Cases

One of the most revealing experiments involved 76 patients who arrived at a Boston emergency room. Both human doctors and an AI system were given identical electronic health records, including vital signs, demographic data, and short nurse descriptions of symptoms.

The results were striking. The AI correctly identified the exact or near-exact diagnosis in 67% of cases, while human doctors were accurate only 50% to 55% of the time.

Researchers said the advantage was especially clear in triage situations where decisions must be made quickly with limited information.

The AI system, based on OpenAI’s o1 reasoning model, improved further when more data was available, reaching 82% accuracy. Human experts scored between 70% and 79% in the same conditions, though the difference was not statistically significant.

Beyond Diagnosis: Treatment Planning Showdown

The study also tested longer-term medical decision-making, including antibiotic treatment plans and end-of-life care strategies.

When asked to evaluate five complex clinical case studies, 46 doctors competed directly against the AI system. The results showed the AI significantly outperformed human physicians, scoring 89% compared to just 34% for doctors using traditional tools such as search engines.

A Life-Saving Insight in a Critical Case

In one case highlighted by researchers, a patient arrived with a pulmonary blood clot and worsening symptoms. Human doctors initially believed prescribed anti-coagulants were failing.

However, the AI detected a missed detail: the patient’s history of lupus suggested an underlying inflammatory cause affecting the lungs. The AI diagnosis was ultimately correct.

Experts Say AI Is Not Replacing Doctors Yet

Despite the strong performance, researchers stressed that emergency physicians are not being replaced.

The study only evaluated AI using structured text-based patient data. It did not account for physical cues such as facial expressions, distress levels, or real-world clinical observation, meaning the AI functioned more like a digital second opinion.

“I don’t think our findings mean that AI replaces doctors,” said Arjun Manrai, one of the lead authors who heads an AI lab at Harvard Medical School. “I think it does mean that we’re witnessing a really profound change in technology that will reshape medicine.”

Dr Adam Rodman, another lead author and physician at Boston’s Beth Israel Deaconess Medical Center, said AI systems are becoming “the most impactful technologies in decades”. He predicted a future “triadic care model … the doctor, the patient, and an artificial intelligence system”.

Growing Adoption and Rising Concerns

AI is already entering hospitals at scale. Nearly one in five US physicians now use AI tools for diagnosis support. In the United Kingdom, 16% of doctors use AI daily and another 15% weekly, with clinical decision-making among the most common applications, according to a Royal College of Physicians survey.

However, concerns are growing over errors and accountability. Doctors in the UK cited AI mistakes and liability risks as major worries.

“There is not a formal framework right now for accountability,” Rodman said, while emphasizing that patients still expect human judgment in life-or-death decisions.

Warnings Over Safety and Oversight

Experts say the study highlights both promise and risk. Prof Ewen Harrison, co-director of the University of Edinburgh’s Centre for Medical Informatics, said the findings show AI is becoming a serious clinical tool rather than just an academic experiment.

“These systems are no longer just passing medical exams or solving artificial test cases. They are starting to look like useful second-opinion tools for clinicians, particularly when it is important to consider a wider range of possible diagnoses and avoid missing something important.”

However, Dr Wei Xing from the University of Sheffield warned that doctors may begin relying too heavily on AI recommendations.

“This tendency could grow more significant as AI becomes more routinely used in clinical settings,” he said, adding that the study did not show whether AI performs worse for certain groups such as elderly patients or non-English speakers.

He also cautioned: “It does not demonstrate that AI is safe for routine clinical use, nor that the public should turn to freely available AI tools as a substitute for medical advice.”

A Future of Shared Decision-Making

With billions being invested in AI healthcare systems, the study underscores a growing shift in medicine rather than a simple replacement of human doctors.

The question now facing hospitals is not whether AI can diagnose better in some cases, but how doctors and machines will share responsibility in some of the most critical decisions in healthcare.