Microsoft’s Innovative Diagnostic AI Model
Microsoft has announced the development of an artificial intelligence (AI) model capable of diagnosing diseases using limited patient information, achieving accuracy levels up to four times greater than those of general practitioners in certain tests. The system, named Diagnostic Orchestrator (DxO), stands out for its ability to mimic the diagnostic procedures employed by physicians during consultations. Unlike other algorithms, DxO does not rely on pre-existing medical reports containing all necessary data to ascertain a patient’s clinical condition.
Instead, the tool begins its assessment with minimal information and subsequently poses questions to gather the individual’s medical history. It determines which laboratory tests are necessary to elucidate symptoms and ultimately arrives at a diagnosis once it accumulates sufficient evidence.
Improving Medical Practice with AI
In a recent article published in the journal Science, the developers assert that their innovation not only promises a more precise application of AI in medical practice but could also alleviate the economic and psychological burdens associated with diagnosis. This is achieved by minimising unnecessary tests and procedures, which can lead to additional costs and anxiety for patients. The authors suggest that as models become more sophisticated, integrating structured sequential inference with clinical experience could revolutionise how AI supports diagnostic reasoning, potentially enhancing the quality and accessibility of healthcare globally.
DxO operates on OpenAI’s GPT-5.5 and is described as an orchestration algorithm. This means it coordinates the reasoning tasks of an underlying model to meet specific objectives. In practice, DxO utilises five AI agents based on OpenAI’s system, with each agent specialising in a distinct function: generating hypotheses, selecting tests, critically reviewing results, managing costs, and verifying the final diagnosis. Together, these components aim to replicate the various stages of clinical reasoning.
Introducing the Sequential Diagnosis Benchmark
The authors have not disclosed the exact dataset used to train the system or the specific training procedures. However, they have highlighted the creation of a new metric to assess its performance: the Sequential Diagnosis Benchmark (SDBench). This tool transforms clinical cases into multi-step interactive encounters.
The assessments conducted using DxO were based on 304 cases from clinicopathological conferences published in The New England Journal of Medicine and 325 emergency department encounters sourced from the Medical Information Mart for Intensive Care IV database in the United States. The former includes cases chosen for their educational and clinical value, while the latter consists of urgent care records, reflecting situations more akin to everyday hospital scenarios.
Evaluating the Effectiveness of DxO
Both datasets encompassed common diseases and rare disorders, along with cases varying in diagnostic difficulty. Each case began with a brief description of the patient’s symptoms and initial circumstances, from which the system had to ascertain what additional information was needed prior to reaching a conclusion. The evaluation considered two key aspects: the accuracy of the final diagnosis and the cost of the complete set of tests requested throughout the process. Consequently, the benchmark measured not only the precision in identifying a disease but also the efficiency with which the necessary information was obtained.
Using SDBench, researchers assessed DxO’s performance and compared its outcomes against those of general practitioners, both in documented clinical cases and emergency care scenarios.
AI Outperforming Human Practitioners
The findings revealed that Microsoft’s system achieved a significantly higher diagnostic accuracy compared to the participating professionals. In complex clinical cases extracted from reports published in The New England Journal of Medicine, DxO attained an accuracy of 83.2% when operating without budgetary constraints. In contrast, general practitioners recorded an average accuracy of 36.1% when they could consult reference materials and 22.3% when working without them.
In emergency scenarios, the tool achieved an accuracy rate of 80.3%, compared to 46.8% for the physicians. These figures pertain to the specific conditions of the experiment and do not necessarily indicate that the system would outperform specialists in all clinical settings.
Balancing Cost and Accuracy in Diagnostics
Furthermore, DxO demonstrated the ability to reduce estimated expenses associated with medical testing, with only a modest decline in its accuracy. In complex cases, when operating under a generous budget, it achieved 83.2% accuracy with an average cost of $5,431 per case. When the budget was restricted to an average of $1,303, its accuracy decreased to 79.6%, reflecting a mere 3.6 percentage points drop despite the overall cost being reduced by approximately 76%.
AI as an Assistive Tool, Not a Replacement
Researchers caution that these results should be interpreted with care, as the experiment was conducted in a controlled environment that does not fully encapsulate the complexities of clinical practice. The tests also do not establish whether the system would maintain the same level of performance when interacting directly with patients or when integrated into routine hospital procedures.
Nevertheless, the authors believe that DxO and SDBench could enhance the work of healthcare professionals by introducing new ways to organise and clarify diagnostic processes, optimise available resources, and mitigate potential biases. However, their real-world utility must be assessed before determining their applicability in health services.
The Future of Medical Intelligence
This research forms part of a broader initiative by Microsoft to develop what the company describes as “medical superintelligence”: an AI capable of surpassing the performance of the finest human-managed healthcare systems. In an article featured in the medical journal BMJ Digital, the company acknowledges that there remains a considerable journey ahead to realise this goal. They argue that improving healthcare requires more than merely achieving superior results to humans in specific tasks; it necessitates the integration of AI capabilities across various stages of medical care and leveraging them collaboratively with professionals.
The authors conclude, “As AI capabilities and their integration into clinical workflows advance, enhancing the quality of care will demand more than superhuman performance in isolated tasks. It will require systems that harness AI, both independently and in collaboration with humans, to surpass the most advanced human-led health systems in integrated medical reasoning throughout the entire care process. To harness its value, it will be essential to understand how these capabilities complement existing medical intelligence and to combine them effectively to improve the quality and accessibility of care.”