Scammers have created deepfakes of some of Australia’s most trusted doctors to sell fraudulent products online. The AMA has had enough.
The AMA has escalated its campaign against AI-generated medical deepfakes directly to prime minister Anthony Albanese, warning that fake doctors are putting patients at risk and eroding trust in legitimate medical advice.
While the association wants tougher regulation and greater accountability from tech platforms, experts say GPs are increasingly on the frontline countering misinformation that reaches patients.
In a letter to Communications Minister Anika Wells last year, the AMA called for a regulatory framework that included a portal for reporting fake or misleading AI content, infringement notices for non-compliance, and a set timeframe for platforms to remove harmful content.
It also called for mandatory ID verification for companies or individuals promoting medical products, and an opt-out mechanism for unsolicited ads.
The AMA doubled down on those demands this week, adding a further call for individuals and companies promoting products be publicly contactable – for example, by email.
For AMA president Dr Danielle McMullen, the issue has become personal.
She discovered a fake social media profile impersonating her after a family member alerted her to it. The profile has yet to be taken down, despite her best attempts.
“Deepfake campaigns that misrepresent trusted clinicians can undermine evidence-based care, spread misinformation about prescription medicines, and encourage consumers to abandon clinically validated therapies in favour of unproven alternatives,” she told media.
“This is not a problem we can ignore or leave to tech companies to solve on their own,” she added in a LinkedIn post.
While the AMA welcomed the establishment of the Office of AI on 15 July, the association urged the government to prioritise its digital duty of care policy.
“We need strong, enforceable regulations that keep pace with the technology. We need accountability from platforms that profit from engagement irrespective of truth. And we need public awareness so Australians can better recognise and report harmful content when they see it,” Dr McMullen said.
Related
Dr Norman Swan, who has been the subject of AI-generated deepfakes mimicking his likeness for almost two years, told The Medical Republic that despite ABC coverage and conversations with the director of security policy and the global head of counter fraud at Meta, effectively nothing had changed.
In these AI-generated deepfake videos, Dr Swan appeared to promote weight-loss products, including the unapproved “all-natural” supplements Glyco Balance and Keto Flow + ACV, which purported to treat heart disease, diabetes and obesity.
In others, he advised people to stop taking metformin, a mainstay of treatment for type 2 diabetes.
“I still get people coming to me asking if I’m really doing that. Quite honestly, I’ve given up,” he said.
Dr Swan said the central issue was that social media algorithms could target patients far more effectively than doctors, while platforms profited from the content and had little incentive to combat misinformation.
“The only reason [Professor Jonathan Shaw] got some traction was that he lawyered up and threatened Meta with a breach of copyright and intellectual property,” he said.
Professor Shaw, an endocrinologist and deputy director of the Baker Heart and Diabetes Institute, was also a victim of fake videos in which he called doctors who prescribed metformin “idiots” and promoted Glyco Balance, claiming it stabilised blood sugar levels “in just 24 hours”.
Dr Swan said coordinated action by the AMA, RACGP, and state and federal governments was essential.
But he also said it was important for GPs to warn patients – via posters, verbally, or on TV screens in the clinic – that any doctor from the practice appearing to sell or promote a product online was a scam, and to report it to the practice if they saw one.
RACGP practice technology and management expert committee member Dr Janice Tan said that in general practice, much of the damage from misinformation was often “cleaned up” weeks or months later, requiring longer consultations. But it shouldn’t be the GP’s burden, she told TMR.
“The cost of this sits within primary care, and the GPs themselves, and the downstream effect is often invisible,” she said.
Borrowing credibility from health professionals could also “fan the flames of general mistrust” towards them, Dr Tan said.
But the greater concern, she said, was that these often-unapproved products were claiming therapeutic benefits – an infringement of the TGA’s advertising guidelines and of Australian consumer law.
“The technology is new, but the principle is the same. Whether it’s AI-generated or a very convincing influencer on TikTok telling you that ashwagandha is helpful for depression or weight loss,” she said.
Dr Tan told TMR that older patients often stopped taking their GLP-1 medication after receiving a convincing WhatsApp message claiming that semaglutide had caused blindness in 1000 people.
“Maybe the failure here is the detection and speed of enforcement, not the absence of enforcement,” she said.
These included platforms taking greater responsibility for their obligations and providing more guidance for clinicians who find themselves the targets of AI deepfakes.
But this also placed greater emphasis on GPs to continue creating a safe environment and maintaining trust with their patients, Dr Tan said.
Specifically, this meant not dismissing or humiliating patients for believing AI-generated content but rather seeking to understand how it could be convincing before debunking it with them, grounded in evidence.
“I’ve had patients who’ve been staunch anti-vaxxers, but over one or two years I’ve managed to convince them to vaccinate their child. And it’s that longitudinal relationship. While we carry a lot of the burden of this misinformation, we’re also best placed to help tackle it,” she said.
Dr Tan said practitioners could refer patients to resources like Health Direct and 1800 Medicare for health questions when not consulting with them.
