Qingyu Chen, PhD, and his team set out to study how artificial intelligence language models are adapted for medicine and found that what these models memorize can be both useful and risky. A model may retain valuable medical knowledge, but in a controlled study using real hospital records, the same fine-tuning—the added training that adapts a model to a specific task—that improved diagnostic performance also made it more likely to reproduce material it had seen during training, including sensitive patient information.

The study, published recently in Nature Communications, reflects a question at the center of Chen’s research: How can medical AI become not only more capable but also more reliable and safer? The study was led by its first author, Anran Li, PhD, who conducted the research as a postdoctoral researcher in Yale’s Department of Biomedical Informatics and Data Science.

Chen is an assistant professor of biomedical informatics and data science at Yale School of Medicine, with a secondary appointment in ophthalmology. He leads research on the accuracy and reasoning of medical language models and on multimodal AI-assisted disease diagnosis, which draws on both text and medical images.