FDA fast-tracks generative AI rollout amid growing safety concerns

Editor's Note

The US Food and Drug Administration (FDA) is deploying generative AI across all its centers by June 30, aiming to accelerate drug reviews and reduce bureaucratic workload even as experts and critics express worries about data security, reliability, and other safety concerns. Medical Design & Outsourcing reported the news May 12.

Launched amid broader efforts to modernize government through automation, the rollout was greenlit after FDA’s first AI-assisted scientific review pilot demonstrated major time savings on repetitive review tasks, the outlet reports. The agency’s new Chief AI Officer, Jeremy Walsh, is overseeing the rapid implementation alongside Sridhar Mantha of the Center for Drug Evaluation and Research (CDER), where internal reviews suggest the technology could save hundreds of thousands of staff hours every year. In fact, one CDER leader cited in the article reports tasks once requiring three days now take minutes with AI supporting human decision-making.

However, the aggressive timeline and lack of transparency have drawn criticism, Medical Design & Outsourcing reports. For example, strategy consultant Markus Brinsa warned on LinkedIn that the FDA appears focused on eliminating "non-productive busywork" without clearly explaining how it will safeguard against AI-induced errors. The article also details how STAT and other outlets have reported increasing concern over “AI hallucinations”—instances where generative models produce false or misleading outputs—which could have dangerous implications in a regulatory setting.

Although the FDA has been in conversation with ChatGPT developer OpenAI about potential collaborations on internal tools such as “cderGPT” and “Research GPT,” no formal agreements have been announced. As detailed in the article, this push toward AI automation coincides with sweeping federal workforce reductions, as the Department of Health and Human Services undergoes staff cuts through Elon Musk’s Department of Government Efficiency (DOGE), eliminating tens of thousands of positions across agencies—including senior FDA leadership. As the June 30 deadline approaches, FDA also has not provided details on how it plans to secure sensitive data or monitor AI performance.