An RACP position statement outlines implementation, use, and monitoring of artificial intelligence tools in clinical practice, but still leaves much to be desired.
“Strength is irrelevant. Resistance is futile.” – Star Trek: The Next Generation
Artificial intelligence in healthcare is here to stay, and the burdens of choosing which tools to use and how to do so safely remain squarely on the shoulders of the individual.
Earlier this year, Professor David Parry, dean of the school of information technology at Murdoch University and AI and eHealth expert, told The Medical Republic that we’re still very much in the Wild West stage with these tools.
Despite some additional frameworks being released since then, this still rings true.
There have been no specific product endorsements from trusted bodies, and the limited government regulation of AI products mostly covers diagnostic tools.
There remains a gap between broad framework principles and practical help for clinicians; it is solely up to each individual user to select, implement, and monitor their AI tools based on the information provided by private companies who stand to profit from their use.
And when have consumers ever suffered from power like that being abused, right?
But the pioneers know the only way out is through, so here is some more guidance, courtesy of the recent Royal Australasian College of Physicians (RACP) position statement, Using artificial intelligence in clinical practice.
There were 11 principles, split into three key areas:
Implementing AI in clinical practice
Before introducing an AI tool, the guidance explained that practices should consider whether it addresses a genuine clinical need, how it will fit into existing workflows, and whether its benefits justify the potential risks and costs.
- Apply ethical principles
AI tools should benefit patients without causing avoidable harm, respect patients’ autonomy, and support equitable access to care.
Additionally, practices should consider whether a tool could disadvantage particular patient groups or distribute its benefits unfairly – although there does not appear to be specifics on how one might do that.
- Involve clinicians and patients
The RACP said clinicians should select their tools based on real clinical needs, rather than introduced simply because the technology is available.
It also stated that healthcare professionals and patients should contribute to all stages of tool development – a principle that’s above the paygrade of the user, but one which will hopefully be taken on board by the government and AI companies.
- Protect privacy and confidentiality
Practices must take reasonable steps to safeguard patient and clinician information used by AI systems. This is mainly about understanding how information is collected, accessed, stored, and shared, and considering whether it is likely to remain protected.
The RACP also highlighted the importance of respecting First Nations data sovereignty principles when handling data relating to Aboriginal and Torres Strait Islander, Māori, and Pasifika peoples.
- Assess value, not just convenience
According to the guidance, an AI product should deliver better or equivalent care without imposing unjustified additional costs. The potential improvements in patient outcomes and practice efficiency need to be weighed against financial costs, implementation demands, and safety risks.
Related
Interacting with AI in clinical practice
Once an AI tool is in implemented into practice, the user remains responsible for interpreting its outputs and making decisions about patient care. The RACP said appropriate training is essential, as is transparency.
- Keep clinical judgement central
AI-generated information should support clinical reasoning and not be treated as a replacement. The position paper advised doctors to check outputs against the patient’s presentation, history, and other relevant evidence instead of assuming that an answer is correct because it appears authoritative.
- Develop the skills needed to use AI safely
Clinicians need education about how their specific AI tools work and where they can fail, and training should be a continuous part of professional development. Using AI to perform a task does not remove the need to maintain the underlying clinical knowledge and skills required to deliver safe care.
- Avoid over-reliance
Clinicians need to remain alert to the risks of automation bias, missed diagnoses, overdiagnosis, and unnecessary treatment. The RACP suggested that AI can be used to challenge or refine a clinician’s initial reasoning, rather than becoming a substitute for independent assessment.
There was an Australian incident reported in September this year in which a dictation/transcription error by an AI medical scribe led to an inaccurate post-operative letter being sent from a urologist to a GP.
The letter falsely claimed that the patient had micro-dosed psychedelic mushrooms and this may have caused her kidney bleeding – a claim that could have affected her workers’ compensation case.
- Be transparent and obtain appropriate consent
Patients should be able to understand when and how AI is being used in their care, with opportunities to opt out where feasible. The position statement identified informed consent as necessary when AI collects personal, identifiable information for practice or training, or directly influences individual patient care.
- Check usability and workflow fit
An AI tool should be straightforward to use and compatible with existing clinical processes. Practices should assess whether it meets expected standards of care without introducing unnecessary work, confusion, or disruption.
Monitoring AI in clinical practice
Introducing an AI tool is not a one-off decision. Its performance, suitability, and effects on patient care and wellbeing need to be reviewed continuously.
- Watch for declining performance
AI output can become less reliable over time, as the information the tools encounter differs from that which they were trained on. The RACP stated that practices need ways to identify and address performance problems before they compromise patient safety, although how specifically to go about this remains unclear.
What can a small practice realistically measure, and who should alert clinicians when a vendor changes a model or its performance?
- Reassess and improve tools continuously
Lastly, another area that is largely out of the hands of users.
The guidance said tools should be reviewed throughout their operational life, with feedback used to identify errors, bias, and opportunities for improvement. Clinicians need a way to report concerns, and developers need to be responsible for problem correction and tool withdrawal.
While monitoring and reporting is an essential focus at an individual level, users aren’t necessarily able to independently audit the model or compel the vendor to fix issues.
The bottom line
The RACP’s principles offer a useful starting point for clinicians, but they do not remove the practical difficulties of choosing and overseeing AI tools. Practices must account for the time and cost involved in researching and evaluating products, integrating them safely into practice, training staff in their use, and monitoring their performance after implementation.
And implementation doesn’t mark the end of the process – once it’s set up, teething issues are resolved, and practices settle into varying degrees of reliance on it, there are still ongoing maintenance costs such as monitoring the tools’ performance and providing continued training as the products evolve.
The central message of the position statement is that these tools require vigilance; active and continued oversight is a requirement for safe use, particularly as AI systems become more autonomous in clinical care.
Finally, there’s no guarantee that the price will remain feasible, data will be protected, or that the tool itself won’t be altered or discontinued.
The RACP’s principles set out what responsible use should look like. The harder question is whether individual practices have the resources and information to put them into effect.
