Microsoft AI's Health team published results in Science on October 22 for MAI-DxO, an orchestrator that runs several model instances as a panel of virtual physicians. Rather than answering once, the system proposes diagnoses, requests the next question or test, then argues over several rounds while weighing each test's likely value against its cost. It was scored on SDBench, which the team built from 304 complex NEJM case records plus typical emergency-department presentations from MIMIC-IV-ED, and which measures diagnostic accuracy, evidence requested and estimated test cost. MAI-DxO reached 83% diagnostic accuracy on the NEJM cases, against 22% for physicians working without reference resources and 36% with them. On the emergency-department cases it reached 80%, against 47% for physicians. A configuration tuned harder for cost held 81% at under half the cost of comparison systems. Microsoft says the gains persisted when the underlying models were swapped for newer ones, including GPT-5.5, without changing the orchestration method, and that it is open-sourcing the benchmark for select NEJM cases, including the Gatekeeper, Judge and cost-estimation code.
Our takeThe interesting variable is not the 83% but the benchmark: SDBench scores the path taken, which tests were ordered and at what cost, rather than a single final answer, and that is exactly where exam-style QA benchmarks fall short. Two caveats for anyone citing the numbers: the evaluation is retrospective by construction, since cases are drawn from published records, and a debating panel multiplies inference calls per case, so the cost comparison depends entirely on how the panel is configured. The open-sourced harness is the part worth reproducing first.