Radiologists may benefit from AI assistance that is shown selectively rather than on every case, according to a study in Scientific Reports. The findings challenge the default of deploying AI uniformly across all reads.
Using data from the public Collab-CXR release, which contains 104,800 pathology-level observations from 20,960 reads by 326 radiologists across 324 cases, the team learned an adaptive assistance policy offline. The policy decides when to display AI predictions based on the algorithm’s confidence, case difficulty and how accurate the AI has been for that reader.
In held-out evaluation, the adaptive policy achieved a mean absolute error of 0.1034 versus 0.1060 for always showing AI, a 2.5% reduction, and 0.1088 for never showing it. The researchers distilled the policy into a three-variable decision tree that matched performance at 93.97% concordance.
Average effects hid the heterogeneity: the confirmatory average treatment effect on diagnostic error was essentially null, which the authors say explains why static AI studies often show mixed results.
They conclude that uniform AI deployment is not the best strategy in radiology and that prospective testing of adaptive, trust-aware support is warranted. The work was funded by King Abdulaziz University in Saudi Arabia.
