Mayo Clinic has detailed the process it uses to evaluate and approve clinical AI tools, with more than 100 applications reviewed so far this year alone. Dr. John Halamka, president of Mayo Clinic Platform, outlined the approach in a post alongside senior research analyst Paul Cerrato.
Executive-led oversight with clinical input
The review regimen evaluates AI complexity, clinical setting, performance and patient safety, user training, workflow integration, privacy and security, and life-cycle management. Every clinical AI application must be reviewed and approved before use by Mayo Clinic staff.
“Even the most sophisticated AI models are no match for years of clinical experience and the ability to sense problems,” Halamka and Cerrato wrote, citing Stanford’s ChatEHR development as a model of clinician-driven AI vetting.
From caution to standard of care
Halamka, who in 2023 warned that generative AI was not yet reliable enough for clinical use, now argues AI “may soon become our standard of care.” But he emphasized that any model deployed in a clinical setting needs rigorous testing, governance and monitoring — with oversight from experienced human clinicians.
