A team at UTS has brought direct brain-computer communication an impressive step closer.
Today some pure science fiction to wrap up this scribe’s Back Page contributions for the year.
Don’t expect too much in-depth analysis from us though. It’s hot, things are winding down ahead of the holidays and a very good boy is in the TMR office distracting us – meet Fred Sheppeard:
On top of that, it’s neuroscience.
What we can tell you is that it’s a non-invasive, portable thought decoder – a program that reads words out of your head without having to install anything made by Elon Musk into your brain, or lie in an MRI, with better results than any previous contender. And it’s being developed just down the road from our office.
Three researchers from the University of Technology Sydney – Distinguished Professor CT Lin, director of the GrapheneX-UTS Human-centric Artificial Intelligence Centre, and PhD candidates Yiqun Duan and Jinzhou Zhou from the Engineering and IT faculty – had their work chosen as the spotlight paper for the prestigious NeurIPS conference held in New Orleans this week.
They team built an AI called DeWave that segments EEG patterns into word-level units and decodes them into text tokens, both with and without eye-tracking – the latter is the real breakthrough, or “unprecedented endeavour” in the team’s words, and makes the tech much more practical to use.
In this study (of which the BP understands maybe one sentence in 10) 29 subjects wearing an EEG cap silently read passages of text, with sentences ranging from short and simple to long and full of abstract concepts.
Here are two examples, where “bold denotes a correct match between ground truth and our prediction. Underline denotes a fuzzy match with similar semantic meanings”.
With eye tracking:
Ground Truth: “The Kid Stays in the Picture” is a great story, terrifically told by the man who wrote it but this Cliff Notes edition is a cheat.
Prediction: The film “says in the Game” is a film about but movie was written, the author who wrote it. also its version is a cheat.
Without eye tracking:
Ground Truth: much of this well-acted but dangerously slow thriller feels like a preamble to a bigger, more complicated story,
Model Output: much of this is-being but not over-. like it disaster-ble to a new and more exciting thing,
It’s a long way from infallible, but it represents a significant improvement on established metrics like the BLEU score. Its BLEU accuracy rating is about 40%, compared with traditional language translation and speech recognition programs, which approach 90%.
You can watch a demonstration, complete with mistakes and lost-in-translation moments, here:
Even at 40% accuracy, this kind of tech offers an extraordinary hope to people who have lost the ability to communicate physically.
It’s enough to leave you lost for words.
And on that note, since TMR will be largely silent from next week for three weeks, this BP scribbler signs out for 2023 wishing you a happy holiday season – see you in the new year.
Send festive story tips to penny@medicalrepublic.com.au.
