Can augmented reality help with Stroke Recovery?
ROLE
UX and UI Designer
Designer Researcher
RESPONSIBILITIES
User Research
Comptetive Analysis
Low Fidelity Wireframes
High Fidelity Mockups
3d asset creation
Illustration, animation and motion design
Business strategy and Roadmap
TEAM (3)
2 Designer Researchers + UX Designers
1 Developer
The Process
What is Wabbit?
Wabbit is an augmented reality mobile game designed for motor skill recovery of stroke patients.
Using Augmented reality and machine learning, Wabbit creates an exercise plan for the patients and converts these into game actions. Wabbit combines the soothing and playful environment of farming, uses augmented reality combined with ambient sounds to create an immersive experience and uses the game actions to help convert the benefits of physical therapy into engaging interactions.
The Client
This project was created for The Comprehensive Stroke Center at Allegheny Health Network to develop a solution for stroke patients in any phase ( before, during or after) with the help of emerging technology.
The Challenge
Stroke is the leading causing of physical disability in the United States. Nearly 80,000 people suffer from a stroke annually causing serious, long-term impairments.
How It Works
The game opens with a list of tasks that it creates for the patient based on physical therapy exercises recommended for them. The game starts with exercises suitable for all patients and modifies them with machine learning based on how the patient responds.
One handed interaction menus help patients with limited mobility navigate easily between tasks lined up for the day and select the one that they like.
A label accompanied by sound pops up to tell the patient on which farm the task is supposed to be done. Sound helps with locating the farm spatially.
An assistive animation is over layered on top of the game action to help them practice the proper exercise form. The line art style helps players still see the other components of the game on a small screen.
A task report is given to the patient at the end of the day making them feel like they accomplished so much by counting every small interaction. In case the patient is not able to do any of the given tasks, the game encourages them that "tomorrow is a new day!" and tells them what they can do and how it will help them.
Especially important is the progress report screen which shows the patient how their motor skills have been improving. This screen can play a major role is keeping patients with no caregivers or support motivated through the recovery period.
Discovery
Stakeholder Mapping
Since Stroke is such a broad subject, and neither of us came from a medical background, we started to make sense of the entire thing by creating a stakeholder mapping. This exercise helped us get an over-arching view of the entire problem space which we could then breakdown into tiny chunks for further analysis and spotting a problem. A further quick study on stroke research made us choose
Stroke Centre
Insurance
Patient ( before, during and after stroke phase)
Existing tools for these phases.
as areas to start in-depth study.
Focussed Analysis and Competitive Research
A study of the chosen areas from the stakeholder maps led to the following insights. During the study, we did competitive analysis of existing products that help with stroke and also went over articles, educational videos and papers related to the subject. Out of this research we figured that we wanted to focus on the recovery phase. This decision was made based on the following insights:
Conceptualization
Addressing Research Insights
To apply all the insights from the research, we decided to do design sprints involving activities such as "Reverse assumptions and 20 questions" ( down left) and crazy eights ( down right) which helped us think very creatively of the ideas. It was out of these crazy eight sessions that I first came up with the idea of "virtual gardening" which the team decided to take forward as a farming game.
Validation
Expert Interviews
Implementation
Designing the Game
As a team with discussed the user flow and wireframes of the entire game and then moved onto making screens for a select task .The task we had chosen was to uproot beats on a farm. The chosen wireframes were then taken into Sketch for high fidelity prototypes, then into Cinema4d & After effects to mock the video and augmented reality section and then into Framer for creating an interactive clickthrough.
An overview of the first user flow where we were trying to figure which are the most crucial screens to convey the idea of the application.
High-fidelity screens
Cinema4d + After Effects for Augmented Reality video
Framer for an interactive prototype
Future Steps
Creating a Roadmap
The current version of Wabbit was designed as being a product that would use machine learning and help patients with a personalized physical therapy using game actions. In the immediate future, Wabbit would like to add specially manufactured objects as mentioned in the question above " how to measure muscle power". Some other things in the pipeline would be to make the product even more immersive with seamless IoT integration and to add social component of being able to play with other stroke patients and to get consultation with an actual doctor based on your gameplay.
Open questions
Why Augmented Reality?
We had 2 main reasons for choosing Augmented Reality:
AR is relatively new technology and hence helps create a novel, exciting experience for a patient. Something that he/she would be very excited to try everyday.
AR allows people to use computer generated objects to interact with with the real world environment which allows for the exercise to be brought to the patients own house. It eliminates the use of traditional screen based gestures for game inputs, and provides a way in which the entire body can be used, if needed.
Why farming?
Ten papers published since 2003 reported positive effects of gardening as a mental health intervention for service users, including reduced symptoms of depression and anxiety. We saw this as the perfect environment for a stroke patient who needed a relaxing immersive environment. We also saw immense potential in coming up with various farm related tasks that involve all parts of the body.
How is muscle power measured?
Wabbit plans to combine the normal gameplay with specially manufactured objects such as squeeze balls, weights and therabands that can give the patients an even more immersive experience of actually holding a tangible object. These objects will also help us record muscle resistance and give the patients some actually resistance that is essential for some exercises.
The same objects would help us in measuring muscle power and reporting it back to the patient for a holistic progress report.
In the first whiteboarding session, we came up with the idea of using objects to simulate the tangible experience.
