Microsoft’s Mustafa Suleyman says its AI outperforms doctors on hard cases as he chases ‘medical superintelligence’

Microsoft published results in Science on Thursday showing that its AI diagnostic system, MAI-DxO (shortened to DxO), correctly diagnosed complex medical cases in a simulation more than twice as often as generalist physicians working individually. Mustafa Suleyman, CEO of Microsoft AI, framed the outcome as movement toward what the company calls medical superintelligence.
DxO is not a single model but an orchestration layer. It coordinates several AI agents that behave like a panel of doctors: they question the patient, order tests, and debate toward a diagnosis. The simulation drew on complex cases from the New England Journal of Medicine. DxO scored 83% correct; primary care doctors and general internists, working alone with reference materials, scored 36%. The comparison excluded specialists and physician teams, which the study notes are the usual handlers of cases at this difficulty.
The study's own numbers indicate that the more powerful underlying models do much of the work, with Microsoft's orchestration adding gains on top. Microsoft tested DxO across models from OpenAI, Google, Anthropic, Meta and DeepSeek; its strongest results paired it with OpenAI's GPT-5 and GPT-5.5.
On cost and accuracy, the picture varies by model. GPT-5 alone, given a short basic prompt, scored 78.9% on complex cases at roughly $8,400 per case in estimated running cost. Adding DxO reduced that cost by about $3,000 while lifting accuracy only about four points. On emergency department cases, neither accuracy nor cost improved meaningfully with DxO versus GPT-5 alone.
GPT-5.5, described as a newer model released in April 2026, split the results. On complex cases DxO raised accuracy from 72% to nearly 87% at slightly lower cost. On emergency department cases, both configurations scored around 85% at similar cost, so the orchestration added little.
The pattern suggests DxO helps most on complex cases and with weaker models. Gemini 2.5 Flash scored 46.4% alone and nearly 69% with DxO. Even so, the distance between weak and strong models was larger than the lift DxO provided. Microsoft researchers said accuracy gains naturally shrink as models approach a theoretical ceiling, and a spokesperson said the orchestration addresses the broader challenge of optimizing both accuracy and cost together.
That dynamic is familiar to anyone who has built agent pipelines. Wrapping a model in a multi-step loop—ask, test, debate—mostly buys better problem decomposition and error correction, which matters when a task has many branching decisions and little when the task is short and well specified. Emergency department cases tend to be more acute and narrower than the diagnostic odysseys drawn from journal case reports, which fits the flat results. In practice, this makes DxO less a replacement for frontier models than a way to squeeze value out of cheaper ones, and a reminder that benchmark headlines can mask where the gain actually originates.
Beyond the paper, Suleyman said Microsoft is working with Mayo Clinic on a healthcare foundation model planned for release next year, which he expects to enable very large leaps. He cast DxO as foundational infrastructure. He also predicted that 2027 will bring something like the coding breakthrough, applied to medical records, with systems streaming accurate predictions about how a record might unfold—foresight he described as valuable to clinicians. He expects healthcare models to improve by an order of magnitude within 24 to 36 months, crediting the flywheel effect of iterating a production model against real-world usage, as happened in chat and coding.
Microsoft is also starting a partnership with Beth Israel Deaconess Medical Center in Boston to evaluate DxO's clinical reasoning across hospital settings. Dominic King, vice president of Health at Microsoft AI, described the design: patients receive standard care—triage, specialist time, treatments, investigations, discharge decisions—and consenting patients also engage with DxO in parallel, with the system's recommendations compared against the human doctors'. King called it likely the biggest study of its kind, spanning emergency and urgent care. Google published a Lancet study with Beth Israel this month evaluating its AMIE conversational diagnostic AI in real-world urgent care visits with physicians monitoring in real time.
The Science paper arrives alongside a Microsoft manifesto defining medical superintelligence as the capacity to surpass the strongest contemporary human-led health systems in medical reasoning and coordination across the full arc of care. That framing moves the goal past task benchmarks such as exam questions and single diagnoses toward integration with clinicians and health systems. Microsoft already sells dictation and radiology tools to clinicians and positions DxO as an expansion of that line; Suleyman said the company must have the leading clinical decision support products in the next few years. He also pointed to healthcare as a possible differentiator against other frontier labs.
Why it matters: Developers building clinical or high-stakes decision tools should read the DxO numbers as evidence that orchestration pays off unevenly—largest on long, ambiguous cases and weaker models, thin on acute settings and frontier models—so evaluation should be stratified by task type rather than averaged. For product teams, Microsoft's roadmap signals a healthcare foundation model with Mayo Clinic next year and a real-world hospital comparison with Beth Israel, which, if it holds up, would move agentic diagnosis from simulations toward deployment questions; that outcome is an open question, not a stated result.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
Microsoft’s new AI diagnostic system beat generalist doctors on hard cases, according to a study in Science. But most of the improvement seems to come from the AI models underneath, not from Microsoft’s system itself.