Roupen Odabashian, Hematology/Oncology Fellow at the Karmanos Cancer Institute, shared a post on LinkedIn:
“Publications alert news from ASCO 2025!
Proud to share that our abstract was accepted for publication at the ASCO Annual Meeting, evaluating artificial intelligence as a clinical decision support (CDS) tool for lung cancer treatment recommendations.
We put GPT-4 and Claude Opus—two advanced large language models—to the test on real-world, de-identified lung cancer cases from our clinic at Karmanos Cancer Institute.
With blinded evaluations by board-certified thoracic oncologists from NCI-designated centers, we looked at:
Accuracy of treatment recommendations.
Comprehensiveness of care (medical, surgical, and supportive).
Presence of hallucinations or potentially harmful advice.
Key takeaways:
– GPT-4 scored significantly higher in accuracy and trustworthiness than Claude.
– But even the best-performing AI generated harmful or inaccurate responses in a notable percentage of cases.
1 in 3 recommendations from both models raised red flags — a clear signal that LLMs aren’t ready for unsupervised clinical deployment.
This study is a step forward in understanding where AI fits—and doesn’t—in oncology care at this stage. It’s a reminder: we need to use these tools carefully, with rigorous validation, human oversight, and constant iteration.
Grateful to our incredible multi-institutional team and mentors across Karmanos Cancer Institute, Mayo Clinic, University of Alabama at Birmingham, and beyond.”
Roupen Odabashian shared research accepted at ASCO 2025 evaluating large language models GPT-4 and Claude Opus as clinical decision support tools for lung cancer. Findings show GPT-4 performed better, but both models occasionally generated unsafe recommendations, underscoring the need for oversight in AI-assisted oncology care.
More posts featuring Roupen Odabashian.