If an AI scribe error is corrected by the doctor, was there ever an error?
Last Thursday, in a packed conference theatre in Birmingham UK, a room full of GPs and digital health leaders heard something candid: an AI scribe, in routine use across dozens of practices, gets things wrong roughly one time in ten.
It was a panel reporting on one of the UK’s biggest AI scribe pilots, which ran for 15 months starting in April 2025, involved 55 practices, used Heidi as the scribe tool, and covered both clinical and non-clinical staff. The pilot transcribed 2.8 million minutes across more than 30,000 consultations.
Nobody flinched in the room apparently when the error rate was mentioned -except for our insider – who sent us a note on it. Â
But then, things apparently got pretty heated on the panel. What started as a throw away line on the standard error rate in the pilot became a point of awkward debate.
Doctors spend a lot of time correcting errors – dictation slips, junior doctor shorthand, the occasional autocorrect disaster, and now, scribing. A wrong line in a consultation note, caught and fixed before the patient leaves the room, has always felt like business as usual.
When the panel volunteered the 10% figure, almost as an aside, it, according to our source, landed as reassurance rather than alarm: the tool isn’t perfect, but we’re watching it.
The problem came when someone started asking why, with the UK regulator in the room, wasn’t anyone freaking out a little more about the figure.
And why not reporting the error rate is considered by everyone, including the regulator, as business as usual.
The problem someone pointed out to the panel is that a GP correcting a hallucinated medication dose is successful human mitigation of a failure that still occurred.
So why isn’t the UK software regulator asking for that error to be consistently reported (or the TGA for that matter)?
Shouldn’t they want to know more specifically what is happening that one in ten times?
If clinicians are spotting nine in ten hallucinations, the system’s safety case rests entirely on an assumption that hasn’t been tested: that the tenth one is rare, mild, and won’t matter.
Nobody in that room in Birmingham could actually say that. And that was decidedly awkward apparently.
If a robot scribe is making errors at a remotely similar rate, in any other regulated industry, that error rate triggers automatic logging, categorisation and reporting. In aviation, pharma and banking the robots flag the anomaly, a human reviews it, and the data feeds a continuously improving safety case.
AI scribes, as far as the public record shows, mostly don’t. The correction happens in the doctor’s head and on the screen, and then it vanishes. There’s no consistent “flag this error” button baked into the functionality.
None of the public material from this major UK ambient voice technology trial reports a structured breakdown of error types, frequency, or clinical risk category. That’s not because the question doesn’t matter. It’s because nobody built the instrument to measure it.
The big question which probably led to the fracas in this session is, why not?
It’s not an engineering problem. A scribe vendor processing tens of thousands of consultations could build a one-tap “this was wrong” button into the editing interface tomorrow.
They could classify error types automatically using the very same AI that generated the note in the first place. They could publish quarterly safety data the way medical device manufacturers are expected to.
If it’s not technical then is it mainly commercial?
An honest, granular error rate would be a marketing problem probably.
Vendors selling adoption, not safety margins, would not be incentivised to surface a number that sounds alarming out of context, even if the underlying risk is genuinely manageable.
Doctors correcting mistakes feels like care.
Regulators counting mistakes is what actually keeps patients safe.
Right now, only one of those things is happening at scale and the gap between them might end up as egg on AI scribe vendor and regulator faces, or… much worse.
Note: an earlier version of this story suggested that the panel presentation on the scribe pilot may have been so heated that people in the debate may have walked out. No one walked out. The debate became quite adversarial and awkward for people watching, as described to us.
