Microsoft launches AI medical diagnostic system Diagnostic Orchestrator, achieving 83% accuracy on complex cases

The research results were published on October 22 in the international academic journal Science. Microsoft compared its self-developed AI diagnostic system with human doctors using a series of medical cases and emergency department data, and established an evaluation benchmark called SDBench. Unlike traditional tests that only require answering a single medical multiple-choice question, this benchmark requires participants to analyze the condition step by step, ask questions, select examination items, and ultimately give a diagnosis, in an attempt to more realistically simulate the diagnostic process of a clinician.

The core idea of Diagnostic Orchestrator is to have multiple AI agents work collaboratively, simulating a consultation team composed of doctors from different professional backgrounds. Based on the patient's symptoms and medical history, the system determines what questions to ask next and what examinations to perform, and continuously adjusts its judgment based on newly obtained information until a final diagnosis is formed. Microsoft hopes to use this to overcome the limitations of a single AI model in complex medical reasoning.

In tests based on hundreds of case reports published in the New England Journal of Medicine, the system's diagnostic accuracy was about 83%. In comparison, general practitioners who could consult reference materials had an accuracy of about 36%, while doctors who could not consult reference materials had an accuracy of about 22%. In another set of cases from the emergency patient database of Beth Israel Deaconess Medical Center in Boston, USA, the AI system achieved an accuracy of 80%, while the human doctors participating in the comparison had an average accuracy of about 47%.

In addition to higher diagnostic accuracy, Microsoft also stated that the system was often able to reach diagnostic results at a lower estimated cost in the tests. One reason is that the system can select examination items based on available information, rather than indiscriminately performing a large number of tests. However, these cost results are based on a unified estimation method used in the study, and do not mean that the system has been proven to directly reduce hospitals' actual operating expenses in real medical environments.

The system is currently powered at the underlying level by OpenAI's GPT-5, but Microsoft emphasizes that what really plays a role is not only the foundation model itself, but also the orchestration mechanism responsible for organizing the diagnostic process, coordinating multiple agents, and verifying reasoning results. The research team stated that even as foundation models continue to be upgraded, multi-agent collaboration may still bring additional benefits. The system was initially developed at a stage when GPT-4.1 was still an advanced model, and subsequent tests also showed that improved model capabilities do not necessarily diminish the value of the collaborative diagnostic architecture.

However, these results still need to be viewed with caution. The tests mainly centered on past cases and selected complex diagnostic cases, and cannot yet prove that the system would achieve the same performance in real clinical environments. Further research is needed to evaluate its reliability when facing ordinary, common symptoms, as well as its performance under real-world conditions such as incomplete information and constantly changing conditions. Microsoft also acknowledges that before the system can truly be put into clinical use, it must undergo rigorous clinical validation.

Suleyman regards this research as an important step for Microsoft toward "medical superintelligence." What Microsoft calls medical superintelligence is not only about enabling AI to surpass human doctors in certain diagnostic tasks, but also about hoping that future AI systems can provide capabilities beyond the traditional healthcare system in multiple key links of medical services.

Dominic King, who is responsible for Microsoft's AI healthcare business, said that diagnosis is only part of the company's healthcare strategy. Microsoft also hopes that AI can help coordinate clinical care, formulate treatment plans, and handle complex issues in medical services involving patient communication and interpersonal interaction. Related research and company articles also explored how AI can take on broader tasks in the healthcare system, rather than merely judging diseases based on symptoms.

Microsoft plans to collaborate with Beth Israel Deaconess Medical Center to conduct tests in a hospital environment and further observe the system's performance in real medical workflows. King said the related work may start in the coming months, but Microsoft has not yet announced a definite timetable. As for commercialization, the company has likewise not given a specific date or product launch plan. The medical field involves strict regulatory, compliance, and patient safety requirements, and the system must complete the necessary validation before it can gradually expand its scope of application.

Suleyman expects that some impressive cases of intelligent systems being actually deployed will emerge in the coming year, and believes that AI will significantly change the way medical services are delivered. However, he emphasized that artificial intelligence does not mean that doctors will disappear from the healthcare system. Doctors' work includes not only diagnosis, but also handling ambiguous information, building trust with patients and their families, and making comprehensive judgments based on specific circumstances. Microsoft hopes that AI can reduce the time doctors spend on repetitive and tedious tasks, allowing them to devote more energy to patient care.

If Microsoft's research results can be validated in real clinical environments, the multi-agent AI diagnostic system could become an important auxiliary tool for doctors dealing with difficult cases and may have the opportunity to improve the efficiency of identifying complex diseases. However, moving from laboratory and past-case testing to daily medical practice still requires clinical validation, regulatory review, and long-term safety assessment. At this stage, this research demonstrates the potential of AI-assisted diagnosis, but it cannot yet be concluded on this basis that artificial intelligence can generally replace doctors.