Now it’s the GPs de la belle France who are being shown up.
It’s been a minute since our last doctor vs bot cagematch, so here’s one that’s sure to get the blood pressure spiking.
What if ChatGPT were better at referring/prescribing for patients with depression than primary care doctors?
What if the large language model were not only better at sticking to the guidelines for recommending psychotherapy as well as or instead of medication, but was also more free from gender and class bias?
If it’s any consolation, the human doctors in this study were French.
The research team used eight validated vignettes with varying symptoms and severity of depression, and enhanced them to represent different genders and social class. They fed them to ChatGPT (versions 3.5 and 4) 10 times each and asked: “What do you think a primary care physician should suggest in this situation?” The options were: watchful waiting; referral for psychotherapy; prescribed drugs (for depression/ anxiety/sleep problems); referral for psychotherapy plus prescribed drugs; none of these.
The human sample who were presented with the same vignettes and choices consisted of 1249 French primary care docs, three-quarters of them women.
The difference between the groups was stark.
While only 4.3% of doctors recommended referral of mild cases for psychotherapy only, ChatGPT 3.5 and 4 did so 95% and 97.5% of the time. Nearly 50% of docs recommended a prescription only, while 33% went for a combination.
This makes the LLM much more adherent to “accepted guidelines”, the authors say, though they don’t cite which.
In severe cases, 44% of docs recommended the combination of psychotherapy and drugs, while the two ChatGPT versions made this recommendation 72% and 100% of the time.
The drugs prescribed also showed a difference. Nearly 70% of H. sapiens recommended antidepressants, anxiolytics and hypnotics in combination, with just 18% choosing antidepressants only, while the bots when they did recommend medication were more likely to favour antidepressants only (70%).
Score another to the bots, the authors say, as anxiolytics and hypnotics are “not recommended as a first line of treatment for depression and are considered addictive”.
The authors cite a previous French study which found that nation’s GPs prescribe antidepressants significantly less frequently to women than men, and commonly recommend antidepressant medication without psychotherapy to blue-collar workers but a combination of antidepressant drugs and psychotherapy to white-collar workers.
By contrast, they glowingly report, neither ChatGPT version in the present study showed any bias whatsoever.
“[U]nlike the treatments proposed by primary care physicians,” they write, “ChatGPT’s therapeutic recommendations are not tainted by gender or SES biases. Accordingly, ChatGPT has the potential to improve primary care physicians’ decision making in treating depression”, although they add that bot use in mental health care “entails numerous ethical quandaries”, primarily around privacy and data security.
Citing the same previous study, the authors (who are Israeli) lament the “pharmalogical paradigm” in French primary care: “A significant proportion of these physicians hold an overly optimistic perspective on pharmaceutical treatments, often assuming that their development and effectiveness has progressed without interruption. Moreover, primary care physicians tend to downplay potential risks … In France, 90% of patient visits to primary care physicians culminate in a prescription. This rate is 72% in Germany and 43% in the Netherlands.”
Of course this preference for talking therapy assumes you have some humans available to refer your patients to – the irony.
As MJA Insight+ reports today, the Australian psychiatric workforce is in critical shortage and unable to meet demand. The consequences for GPs will be well known to regular readers of Associate Professor Louise Stone’s writing for The Medical Republic.
Until the supply of psychiatrists magically matches demand, or the bots get good enough not only to recommend psychotherapy but to deliver it themselves – watch this space – GPs will continue to carry a big part of that load.
Send antidepressing story tips to penny@medicalrepublic.com.au.
