Nine widely used AI models were asked more than 8,500 clinical dosing questions spanning over 2,200 FDA-approved drugs. Around one answer in fifty carried an error, and Doximity classified 43% of those as significant and 3.4% as potentially lethal.
The company released the benchmark alongside DoseCheck, a free verification layer bolted onto Doximity Ask, its clinical assistant for physicians. Doximity says the checker returns correct dosing findings 99.9% of the time. It claims its platform reaches more than 165 health systems.
The misses clustered around a specific failure. Models routinely skipped dose adjustments for age, kidney function and drug interactions even when the prompt supplied that information. Recently approved drugs carried five times the odds of an error, a gap Doximity attributes to thin safety data on newer molecules.
That pattern matters because dosing is where an AI shortcut becomes an injury. Roughly 81% of physicians now report using AI in practice, more than double the 2023 rate, per the American Medical Association. Most are using general-purpose chatbots, not validated clinical tools.
Doximity is not a neutral referee. The benchmark doubles as marketing in a crowded field, arriving as OpenAI and Anthropic push into healthcare and rivals publish competing accuracy claims. Doximity recently publicized Stanford and Harvard research finding specialized clinical AI beat general-purpose models. Another team published the opposite result this summer.
The portable finding is the failure mode. Doximity’s chief AI researcher, Dr. Louis-Antoine Mullie, said the same weaknesses surface elsewhere in clinical care through stale knowledge and missing patient context.
An answer that omits the patient in front of you is a guess wearing a lab coat.
