Verbal Spontaneity – Personality Retention
Dr Peter Scott-Morgan was diagnosed with Motor Neuron Disease (MND) in November 2017 — a disease that attacks the neurons that send signals to the muscles to move. As the disease progresses, all muscles are paralyzed except for the eyes and heart.
While doing research online, Peter discovered that Intel developed the Assistive Context-Aware Toolkit (ACAT) with famous MND sufferer, Stephen Hawking. ACAT is free, open-source software to help people with extreme disabilities to communicate through an interface with a digital keyboard. ACAT also helps people to accomplish tasks with their computers, like writing documents, accessing the internet and emails. Stephen Hawking used a sensor attached to his glasses that detected minor moves in his cheek to select commands in ACAT.
Our design challenge was to help people with extreme physical disabilities to retain communication and personality. Fjord’s role was to redesign the front end, enabling it to interface with several new technologies (eye tracking, avatar, speech synthesis and AI Predictive text), and to overhaul the UI and UX to make it faster, more intelligent, more comfortable and easier to use. Our timeframe was just three months with a small team (Fjord and Intel) of two developers, one UX researcher/designer, two directors and two project managers. Half of the team worked on their own time. To increase complexity, the budget was very limited, and we had numerous technical constraints from working with software built in the 1980s.
We created a new, faster and more comfortable interface for ACAT. We collaborated with Intel to overhaul the system’s UX and UI, updating it so that it can now be controlled by eye movement, using readily available and affordable hardware. We designed a new circular keyboard and improved the predictive text, adding AI sentence generation to make it faster. Based on our research insight we included a unique situational phrasebook that responds to time, location, environment and specific individuals. We added patient speech synthesis created by Cereproc and virtual avatar software created by Optimize3d and Embody Digital to the interface.
After five prototype iterations, Peter is using the interface every day. He’s found he can use it for longer periods of time without headaches or eye strain. It’s measurably faster and he’s improving his speed as he continues to use it. Having now completely lost his voice, it’s the only way Peter can communicate. Being able to use and hear his own voice is “magical”.
Our solution will be available as a free software download. Other assistive technologies are often prohibitively expensive, but this solution and the hardware (roughly £100 for the eye-tracker) is something that people with limited economic means can use to continue communicating, even as their conditions progressively limit their capacity to interact themselves. Despite the use of highly advanced technology, this is a human-centred design project. Technology is used as a medium for the benefit of people to continue thriving in their day-to-day life.
We are humbled that Verbal Spontaneity – Personality Retention (Peter 2.0),
- won a bronze Health & Wellness Cannes Lion Award 2021.
- won MWC’s GLOMO Awards 2021 for the Best Mobile Innovation for the Connected Human category.
- honorable mention in Fast Company’s Innovation by Design 2020 Health category.
- shortlisted for D&AD’s 2020 Product Design, Prototypes category
- won a D&AD’s 2020 Future Impact Design Pencil