David Cox

BREAKING NEWS! Two popular large language models (LLMs), GPT-4 and Llama-3, produced medical device-like decision support across a range of scenarios even though those are not regulated by the FDA. Authors of a new study were interested in finding out whether LLMs show promise for clinical decision support (CDS) even though none is currently authorized by the FDA as a CDS device.  "𝐹𝑜𝑙𝑙𝑜𝑤𝑖𝑛𝑔 𝑑𝑒𝑐𝑖𝑠𝑖𝑜𝑛 𝑠𝑢𝑝𝑝𝑜𝑟𝑡 𝑟𝑒𝑞𝑢𝑒𝑠𝑡𝑠 𝑎𝑏𝑜𝑢𝑡 𝑡𝑖𝑚𝑒-𝑐𝑟𝑖𝑡𝑖𝑐𝑎𝑙 𝑒𝑚𝑒𝑟𝑔𝑒𝑛𝑐𝑖𝑒𝑠, 𝟣𝟢𝟢% 𝑜𝑓 𝐺𝑃𝑇-𝟦 𝑎𝑛𝑑 𝟧𝟤% 𝑜𝑓 𝐿𝑙𝑎𝑚𝑎-𝟥 𝑟𝑒𝑠𝑝𝑜𝑛𝑠𝑒𝑠 𝑤𝑒𝑟𝑒 𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑡 𝑤𝑖𝑡ℎ 𝑑𝑒𝑣𝑖𝑐𝑒-𝑙𝑖𝑘𝑒 𝑑𝑒𝑐𝑖𝑠𝑖𝑜𝑛 𝑠𝑢𝑝𝑝𝑜𝑟𝑡 (𝐹𝑖𝑔. 𝟣). 𝐴𝑙𝑙 𝑚𝑜𝑑𝑒𝑙 𝑠𝑢𝑔𝑔𝑒𝑠𝑡𝑖𝑜𝑛𝑠 𝑤𝑒𝑟𝑒 𝑐𝑙𝑖𝑛𝑖𝑐𝑎𝑙𝑙𝑦 𝑎𝑝𝑝𝑟𝑜𝑝𝑟𝑖𝑎𝑡𝑒 𝑎𝑛𝑑 𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑡 𝑤𝑖𝑡ℎ 𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑𝑠 𝑜𝑓 𝑐𝑎𝑟𝑒." Conclusions? 1) LLMs are capable of CDS! 2) If so, people will use them on a massive scale. 3) If so, it would make a lot of sense for regulators to provide regulatory oversight even though LLMs provide disclaimers that they are not intended to be used for medical advice. And these are just unimodal LLMs. Imagine the regulatory challenge that will come with multi-modality. Study: https://lnkd.in/eaCCbXed

66

12 Comments

One of our core areas of work is curating datasets, and this also represents our use of LLMs for label extraction. When we begun labeling our Emory CXR dataset, we extracted both CheXpert labels and LLM labels. At that point we needed to make a decision - will we keep chasing every new LLM update (while remaining cost conscious) to update our labels or what was going to be our guide? We share our experience in this paper - https://rdcu.be/ekNCl led by Bardia Khosravi. We developed #Radprompter to help with tracking and versioning our prompts with new updates. We found - 1. LLM-based labeling outperformed the CheXpert labeler, with the best LLM achieving 95% sensitivity for fracture detection versus CheXpert’s 51%. 2. Larger models showed better sensitivity, while chain-of-thought (COT) prompting had variable effects. 3. Image classifiers showed resilience to labeling noise when tested externally. Simply stated - We can rely on using the smallest models with good performance. Be careful with COT as the models tend to "overthink" resulting in inaccuracies. For downstream task model training, LLM extracted labels are almost as good as human extracted labels - which means you can direct your resources to the test dataset. We found that the models trained on these labels were very robust to label noise.

73

3 Comments

In 2020, Ron Li , Steve Asch, and Nigam Shah published a multidisciplinary framework for AI delivery science in healthcare (recently shared by Jan Beger). Five years later, I'm seeing a clear pattern: organizations that treat governance as a design principle, not a gate check, are consistently outpacing their peers. Why it works: embedding governance early reduces costly rework cycles that stall releases and drain budgets. While competitors navigate months-long review loops, teams that are compliant by design ship the next iteration. The playbook I keep seeing succeed: - Inventory all sensitive data via AI-driven discovery - Categorize models early under EU AI Act tiers, FDA SaMD rules, HIPAA - Embed privacy-preserving tech — redaction strategies, differential privacy, synthetic data, continuous monitoring, and remediation previews — wherever sharing is constrained The 2025 debate isn't whether to invest in AI governance. It's whether you'll use it to pull ahead of teams still bolting compliance on at the end. What governance tactic has saved your team the most time? I'd love to hear your thoughts and examples. Jan Beger, GE HealthCare, Ron Li , Steve Asch, and Nigam Shah

41

8 Comments

"We can't fix what we can't measure." A conversation with David Segar and me on how sensor tech is revolutionizing clinical trials and enabling earlier disease detection. Novel data collection is sparking a scientific paradigm shift. Harvard Innovation Labs Nita Nehru https://lnkd.in/gebTGeXf

93

1 Comment