- Ralph Johnstone
Ingenuity and heart help improve lives
Some people have imaginations as big as the universe. We marvel at Edwin Hubble, who exploded our world-view a century ago with the discovery that our Milky Way was one of multiple galaxies. Or Stephen Hawking, who dared us to believe there may be other universes beyond our own. And then there are the literary visionaries like Isaac Asimov, who introduced us to the idea of friendly, knowledge-gathering robots decades before AI arrived in our lives.
But once in a while, a more earthly futurist lands among us: a scientist with the brains to build something ingenious and the heart to use it to improve others’ lives.
Pioneering future-tech
Dr Jordan Nguyen fits into this category. The ebullient Sydneysider is widely known as a future-tech presenter, drawing rave reviews at global expos with his mind-bending predictions about our future. In 2010 he made his first appearance in the Australian media, when as part of his PhD, he built a system to drive a wheelchair with the ‘brainwaves’ of electroencephalography (EEG).
But back then, the world wasn’t quite ready for Dr Nguyen. Six years later, he presented a special double-episode of ABC’s Catalyst, helping a young boy with cerebral palsy drive a buggy with his eye movements. Although the show won a major documentary award, the world still hadn’t totally woken up to the possibilities of Jordan’s technology.
But the high-speed engineer was not slowing down. While the world got on with its daily busyness, Jordan was building on the realisation that AI and robotics can not only drive wheelchairs and cars – they can completely re-map the road for people with severe or profound disabilities.
Revolutionising Accessibility
In the next six months, the world will start hearing a lot more about Jordan Nguyen. In November, with funding from the Telstra Foundation, his team at Psykinetic are due to complete the production prototype of a device called AURA – a powerful tablet computer unlike anything developed for people with disability in the past.
With a prodigious processor, a 1200-nit screen, and a design built with a deep empathy for those who struggle with communication or mobility, AURA holds a very real potential to revolutionise people’s lives. Its ‘brain’ is a software platform called StarGaze, which will carry a range of apps and plugins specially designed to be operated with eye-trackers, push and muscle switches, brain-computer interfaces, or other tech used by people with profound disabilities. These apps will not only enable their users to write and speak with their eyes, but to draw, edit videos, browse the web, write code, build 3D CAD models – basically, says Jordan, any tech you might need for improving your life.
“This platform is so much more than playing games and communicating,” says Jordan. “It’s having a job, studying, growing your skills, being able to contribute to society in the way that works best for you.”
Thanks to the Telstra Foundation, AURA will aim to open up a world of new digital opportunities for our friends and the communities we’ve been designing this technology with.
Mad scientist, sane science
Jordan Nguyen’s laboratory in the Sydney suburbs is like a mad scientist’s workshop. Robots in various states of mechanical undress, circuit boards, 3D printers, teetering piles of tools. Pride of place, parked in the centre, is Jordan’s silver DeLorean sports car, with its trademark gull-wing doors. Jordan must be fed up with comparisons to Back to the Future, but he fields them with his usual good grace.
“I was lucky enough to know the legendary Australian artist, Ron Cobb, who was responsible for conceptualising Doc Brown’s DeLorean as a time machine,” he says. “So when I had the chance to buy a DeLorean… well, it was the one thing I’d coveted most in my life!”
If you’re lucky enough to get on a call with Jordan in his workshop, chances are you’ll get the tour. And be introduced to Koobo (a portmanteau of ‘book’ and ‘robot’), a companion robot which Jordan initially built for his beloved grandmother Shirley, who passed away before he could hand him over. Like thousands of seniors, Shirley was stuck in an aged care home during COVID, with her family only able to visit her for an hour a day – and then, suddenly, not at all.
“She had no roommate, and no TV; she was very lonely,” says Jordan, with a rare crack in his always-there smile. “So I thought, what if I could build a robot who could keep her company and chat about all the things she liked talking about? That was going to be Koobo’s job.”
Koobo: a YouTube star on the rise
Today, Koobo plays a starring role in Jordan’s new YouTube series, Enter the Future, accompanying his master – R2D2-style, in the DeLorean, of course – on a tour of the landmarks of our future. And he’s doing so with a new body, a powerful processor, a sophisticated suite of sensors, and the kind of affability that would make Dr Asimov proud.
“He’s funny and kind, and he can hold up conversations with a group of people in a very natural way,” says Jordan. This will undoubtedly make Koobo a useful companion when they call in on Shirley’s retirement home, where Jordan wants to take him to meet some of his granny’s friends. “The way I see it, he’s part of her legacy – so we’re doing this trip for her.”
Jordan’s own kind-heartedness was born of a loving Vietnamese-Australian home, and a few timely brushes with providence. It’s well known that, while at university, he narrowly avoided serious injury diving into the bottom of a backyard swimming pool. A few weeks of bad neck pain started him thinking about living with a disability – and pretty soon, he was putting his coding to practice on his famous “mind-controlled wheelchair”. (He also started corresponding with Dr Hawking – but that’s another story.)
The eyes have it
In 2014 Jordan set up Psykinetic, which started building on research suggesting that eye movements could provide a more practical and reliable way of interfacing with computers than harder-to-measure EEG brainwaves. “We saw this repeatedly, with people with stroke, cerebral palsy, Parkinson’s and MND,” says Jordan, “that the eyes offer a more practical way to control a computer, which is relatively simple to set up, and can work in conjunction with other sensors and switches that people may already use.”
In 2016, using Psykinetic’s eye-tracking technology, a music-making app called Atmosphere had its first public outing at the Sydney Opera House, where a brilliant young artist called Jess Irwin played a celebrated concert with the Australia Piano Quartet using her eyes to play different keys. That piece of tech will be one of the first apps to be publicly available on StarGaze.
Over the past eight years, Jess, who lives with a high level of cerebral palsy, has gone from “painfully slow” hand-typing, to speaking at three times the speed through Psykinetic’s Frontier eye-typing software. As Psykinetic’s DevOps Engineer, she oversees a development program that includes everything from cognition-building games for children to 3D modelling of customised accessibility aids – all with her eyes.
“AURA will open up so many opportunities for non-verbal people like me,” says Jess. “I can see a brand new world opening up for so many people.”
Support from Telstra Foundation
Last year, the AURA program gained traction with a development grant from Telstra Foundation, which enabled Jordan, Jess, and Psykinetic CTO Nick Temple to produce their first wheelchair-mounted prototype for testing with a small group of users who rely on eye-tracking and sensor technologies.
This year, the program has gained further support from the foundation, whose board has been moved by the real potential of this Australian invention to change lives globally. “We believe that this technology will be a gamechanger,” says Telstra Foundation’s CEO, Jackie Coates, “and we’re so proud to be part of the journey to get this to market.”
Affordable accessibility
Psykinetic’s technology addresses several of the Foundation’s key priorities, including improving digital skills, access and connectivity for vulnerable Australians aged between five and 25. The fact that Jordan Nguyen is prioritising AURA’s low-cost production, and will include an impressive range of StarGaze apps to users, is a major boon in a market renowned for its excessive hardware costs.
“I have friends with profound disabilities who have to get their carers to bring in a new device when they shift between writing and playing games,” says Jordan. “But that’s only if they can access them, because many of the upper-end devices cost upwards of $15,000.”
With its latest grant, Psykinetic have been fine-tuning the AURA prototype with the award-winning design studio, Design+Industry, which is helping them strip back their electronics costs and design a version more suited to manufacturing at scale. The prototype demonstrates an exceptional understanding of users’ lives, with its “double brightness” enabling use in direct sunlight, and surprise features that promise unprecedented opportunities for face-to-face interactions.
Pushing the boundaries
Over the coming months, the Psykinetic team will also be building out the backend of StarGaze, and trialling their apps with people with various disabilities and accessibility tools. Their proprietary eye-tracking algorithms, meanwhile, remain a powerful motivator. Researchers have traditionally placed the maximum number of words a person can type with their eyes at 13-15 per minute. Over recent months, the wizards at Psykinetic have more than tripled that number.
“Nick’s now up to 48 words a minute, but we have a few mates who are chasing him down,” laughs Jordan, who admits that his team and their friends are “super-competitive”.
The lightning-fast speeds available through Frontier stem from an inspired algorithm that combines speed and word recognition technologies with advanced estimation of users’ intentions – a method that’s so ingenious, and will become so valuable in the market, the team have had to patent it. “Our system’s designed to adapt and improve as you get used to it,” explains Jordan, “so no matter how fast you can type with your eyes, the computer will always keep up.”
A collaborative network
Today, Psykinetic’s growing stable of apps includes Frontier, Atmosphere, a Calculator with core mathematical functions, a spaceship-piloting game called Kinetic Zero, and a basket of Tabletop Tools that will enable children – often for the first time – to roll a dice, flip a coin, or throw a hand in scissors, paper, rock. These apps may be basic, but the foundations they are laying promise a world of difference to people who traditionally have not had the same level of access to social and professional settings that many of us take for granted.
“Seeing Jess type so fast with her eyes using the tech we’ve developed together is such an incredible experience,” says Jordan. “But to know that we now have the chance to scale this project to potentially become a tool for thousands of people to express themselves with and use to take part in new pastimes and projects – this is what really drives us.”
Empowering innovation
The other driving factor, which shows Psykinetic as a true force for good, is the company’s determination to radically lift collaboration across the market. To do this, they’re developing a series of ‘software development kits’ that will provide foundational code – or APIs – free of charge to any developer or studio that wants to develop an accessible app – in the hope that when they complete it, they’ll host it on StarGaze. Psykinetic has already established partnerships with a handful of studios that are building games to help non-verbal children improve their cognition and develop eye-tracking skills.
“Most inclusive apps have proprietary launchers, installation processes or hardware, and writing code specific to each vendor can be a huge barrier for developers who want to make an inclusive app,” explains Nick Temple. “Making StarGaze a thriving platform will only be possible by teaming up with the amazing companies that are already producing software and technology in this space – while making it as simple as possible for developers to make their apps more inclusive.”
Fostering collaboration
“In the past, great accessibility innovations have tended to come from individuals developing small projects to help a friend or family member,” adds Jordan. “We want to give these innovators the tools to build something that can scale for anyone who might be able to use it. That’s the dream: to build a collaborative network linking inclusive developers and studios that are invested in accessibility – and helping them share their innovations to make the world more accessible for everyone.”
Read more about Psykinetic’s technology at: Psykinetic – Products; and follow Dr Jordan and Koobo’s adventures at: Dr Jordan’s Youtube channel.
