Study suggests physician’s medical decisions benefit from chatbot – A study showed that chatbots alone outperformed doctors when making nuanced clinical decisions, but when supported by artificial intelligence, doctors performed as well as the chatbots.
https://med.stanford.edu/news/all-news/2025/02/physician-decision-chatbot.html
4 Comments
From the article
>Artificial intelligence-powered chatbots are getting pretty good at diagnosing some diseases, even when they are complex. But how do chatbots do when guiding treatment and care after the diagnosis? For example, how long before surgery should a patient stop taking prescribed blood thinners? Should a patient’s treatment protocol change if they’ve had adverse reactions to similar drugs in the past? These sorts of questions don’t have a textbook right or wrong answer — it’s up to physicians to use their judgment.
>[Jonathan H. Chen](https://profiles.stanford.edu/jonc101), MD, PhD, assistant professor of medicine, and a team of researchers are exploring whether chatbots, a type of large language model, or LLM, can effectively answer such nuanced questions, and whether physicians supported by chatbots perform better.
>The answers, it turns out, are yes and yes. The research team tested how a chatbot performed when faced with a variety of clinical crossroads. A chatbot on its own outperformed doctors who could access only an internet search and medical references, but armed with their own LLM, the doctors, from multiple regions and institutions across the United States, kept up with the chatbots.
LLMs are good at knowledge breadth, not so good at knowledge depth. Diagnosis is often about knowing what patterns to look for, which is what AI is good at.
Picturing the bot playing devil’s chatvocate going like “what if it’s Lupus” or sth
GREAT…. a highly skilled human in one of the hardest jobs we can do, is ALMOST AS GOOD AS A CHAT BOT….
makes me feel great.