When the Human in the Loop Is the Problem

The medical establishment has a reflex that sounds like common sense: when AI helps with a diagnosis, a human doctor should make the final call. A new viewpoint in the Journal of the American Medical Association argues that this reflex is about to become dangerous.

The piece, authored by bioethicist Ezekiel Emanuel and AI health startup CEO Neal Khosla among others, makes a pointed prediction: by 2030, autonomous AI will outperform any human-AI team at medical reasoning. If regulators lock in mandatory human-in-the-loop requirements now, the authors argue, they will be legislating an inferior standard of care into place just as the technology surpasses it.

The evidence that AI is already ahead

The JAMA authors build their case on two arguments. The first is empirical: since 2024, a growing body of research shows AI matching or beating doctors across the five core reasoning tasks of medicine -- taking a patient history, making diagnoses, choosing tests, treating according to guidelines, and managing chronic disease.

One study found that ChatGPT o3 named the correct diagnosis first 60 percent of the time across 377 complex simulated cases, compared with 15.9 percent for 20 internists. Microsoft's AI found the correct diagnosis under budget constraints roughly four times as often as doctors, at lower cost. The authors dismiss contrary studies as outdated or methodologically weak, often because they excluded the best available models.

The second argument is a forecast. AI models are improving fast, while doctors who lean on these tools lose their own diagnostic skills through reduced practice -- a phenomenon known as automation atrophy. Once the machine is clearly ahead, the doctor doing the checking turns from a safety net into a source of error.

A meta-analysis of 106 experiments backs this up. When the human is the better performer, the combination helps. When the AI is better, the human makes the result worse by overruling the system in the wrong places. In one study, GPT-4 alone scored 92 percent on diagnostic reasoning, while doctors with access to the same model scored just 76 percent.

The authors draw a parallel to chess. After Deep Blue beat Garry Kasparov in 1997, human-machine teams dominated for years. By 2017, AI had become so strong that even the best human-machine teams could no longer compete. Medicine, they argue, is following the same trajectory.

The conflict of interest that complicates the message

The viewpoint carries an unusual disclosure. Emanuel is a heavyweight in US health policy and one of the architects of the Affordable Care Act. Khosla is CEO of Curai Health, an AI telemedicine company whose business model depends on autonomous AI. His father, Vinod Khosla, is an investor in both OpenAI and Curai Health.

This does not invalidate the argument, but it means two of the authors stand to benefit directly from the regulatory future the piece champions. The Decoder, which first reported on the JAMA viewpoint, noted this tension. Medical professor Robert Wachter, author of The Digital Doctor, has called AI-only care the "economy class" of medicine -- a framing the JAMA authors explicitly reject as unproven.

The limits the authors acknowledge

The JAMA viewpoint is not an unqualified endorsement of autonomous AI. The authors concede that nearly all the evidence comes from simulations of single tasks, not from real patient care. The handoff of information between human and machine is a known weak point. Physical procedures like surgery, childbirth, and colonoscopies will remain with humans for now, as the robotics are not yet adequate.

Autonomous systems also fail in ways that humans do not. Hallucinations, internet outages, and cyberattacks are risks that must be weighed against higher accuracy rates. The authors do not dismiss these concerns -- they acknowledge them, and argue that the balance of risk still favors autonomous AI for cognitive tasks.

What this means for regulation

The piece arrives at a critical moment. Regulators in the US and Europe are actively drafting rules for AI in healthcare, and the default assumption in almost every framework is that a human must remain in the loop. The JAMA authors argue this is a mistake that could cement a form of care that will soon be obsolete.

Liability, payment models, medical training, and licensing all need to be rethought, they argue. If a doctor is no longer the best person to make a diagnosis, the legal and financial structures that assume a doctor is always in charge become obstacles rather than safeguards.

The question the piece leaves open is the hardest one: how do you transition from a system designed around human judgment to one that trusts AI more, when the evidence is still drawn from simulations and the failure modes are still poorly understood? The JAMA authors have a clear answer -- start now, before the gap between what AI can do and what regulation allows grows even wider.

Sources

• The Decoder: As AI beats doctors, regulators shouldn't force a human into the loop, JAMA piece says
• Forbes: Autonomous AI Should Replace Human Doctors, Billionaire's Son Argues
• Medical Economics: When will human physicians start dragging down AI in health care?
• Endpoints News: AI will be better than doctors at some medical tasks, healthcare and VC experts predict
• Penn LDI: How the U.S. Can Regulate AI Without Stifling Health Care Innovation