Paper admitted to the ASSETS '24 Conference
An exploratory study that investigates the potential applications of generative AI in enhancing the workflow of occupational therapists for creating and adapting assistive technology solutions for people who live with limited hand mobility.
Find the GAPs ...
The assistive technology (AT) adaptation process includes stages such as clinical assessment, searching for off-the-shelf ATs, adaptation, AT evaluation, and further adjustment.
GAP 1
​
Customization
Existing products offer limited design options and are mostly one-size-fits-all. Cost is another critical factor that occupational therapists (OTs) need to consider when selecting devices.
GAP 2
​
Creativity
Relying solely on human thinking can lead to limitations in generating novel and stylish design concepts.
GAP 3
​
Efficiency
OTs may not always have sketching skills, so they often prefer hands-on prototyping. However, this approach can take a lot of time as they experiment and test various designs to find the right solution.
GAP 4
​
Communication
Clients often lack a clear visual concept of the adapted products they will use, which can reduce communication efficiency and make them less engaged in the design process.
GAP 5
​
Reproduction
It requires significant time and resources to explore various solutions, and complex adaptations are challenging to reproduce if they break.
AI can BRIDGE the GAP!
Text-to-image AI generator
Quickly generate diverse concepts based on OT's input and inquiries.
Image-to-3D AI generator
Assist OTs in turning generated concepts into tangible outcomes.
Midjourney and Meshy are selected as the primary AI generators and integrated their official APIs into Thera AI.
Midjourney's extensive repository enhances the feasibility of generated products, while also delivering the highest image quality.
Meshy produces highly accurate 3D models and can generate them within just 1 to 2 minutes.
AI is useful, but ...
It takes time for novice users to come up with suitable prompts, and the process often involves trail and error to find the right words. Moreover, achieving precise, realistic images requires familiarity with professional design terminology.
Use
Keywords!
to craft precise prompts efficiently.
The Selections of Keywords
Target User
Product Type Specification
Selected keywords include "Aid device," "Adaptive technology," and "Assistive technology," covering a range of products designed for various assistive needs.
Keywords such as "Designed for the elderly" and "Designed for people with limited hand mobility" ensure that the products are tailored to specific user groups with varying levels of hand function.
cat.
0 1
cat.
0 2
cat.
0 3
Activities of Daily Living (ADLs)
Keywords such as "bathing," "dressing," and "feeding" were used to identify hand-related ADLs that need improvement. Sub-categories like "spoon," "weighted grip," and "extended grip" further specify specific utensils and adaptation types to better address user needs and functionality.
cat.
0 4
cat.
0 5
cat.
0 6
Over 80 prompts were tested using terms commonly used by OTs in daily practice and were grouped into 6 categories.
Custom Keyword Library
Material & Finish
OTs can add, edit, and delete keywords to create a personalized keyword library for quick prompt composition.
Image Styles
Keywords such as "grip with textured surface" were included to add texture and improve usability for weak grip strength.
Keywords like "precisionism," "photo," "perspective view," and "white background" were selected to produce clear images for 3D modeling.
Interface Prototype
UI components - text-to-image tab
Free text entry box
Image display
Image download
Image generation button
Keywords function
UI components - image-to-3D tab
Image upload
3D model download
3D model preview window
3D model generation button
Usability Test
40 to 70 minutes
40 to 70 minutes testing session
5 OTs
Procedures
Prior to the workshop, the OTs completed an AT User Profile Form to create a hypothetical client profile. The form included information such as the user's age, gender, diagnosis, and functional limitations. This form served as a reference for the OTs to create suitable ATs using GenAI according to the user's needs and impairment type.
During the workshop, the OTs used text-to-image AI generator to generate visual representations of the desired ATs, and then employed the 3D AI generator to produce 3D models from the generated images.
Following these activities, a semi-structured interview was conducted to collect their insights about the use of GenAI in the AT adaptation process in their clinical practice.
assistive device,
aid designed for the elderly,
designed for people who have limited hand mobility, designed for people who have limited hand strength, grip with textured surface, photo, white background, perspective view,
cup,
holder,
two handles on the side,
lidded cup
adaptive technology,
designed for people who have limited hand strength,
aid designed for the elderly,
cutting knife,
angled utensil,
holder,
wood,
precisionism,
perspective view,
white background
kitchen aid for cooking,
assistive device,
aid designed for the elderly,
designed for people who have limited hand mobility,
designed for people who have limited hand strength,
wider grip,
grip with textured surface,
photo,
rocker knife,
white background,
perspective view
adaptive feeding tool for a patient with limited grip and finger function with Velcro strap to secure around hand or wrist, adaptive technology, assistive product, designed for people who have limited hand strength,
designed for people who have limited hand mobility,
thickened grip,
hand grip,
grip with textured surface,
white background,
weighted grip,
top view,
fork
typing aid,
assistive device,
universal cuff,
plastic, photo,
perspective view,
stylus
Generated images and printed 3D model
Prompts and generated images.
Generated images and printed 3D model.
OT's prospective on AI integration and workflow in clinical practice
Limitations of current AI generator
Not suitable for complex adaptations
"Sometimes it just might require someone to custom fit something to a patient...like a splint where you have to adjust their wrist, finger, and knuckles positioning."
Missing certain commands in the prompt
"frustration when AI-generated images do not meet all specified keywords in the prompt"
Requires OT professional expertise to evaluate usefulness
"AI prompt composition and the usefulness of generated ATs still require an OT's eyes on it...it's important to ensure there is no potential for skin breakdown or harm."
Interface insights
Interface suggestions
Keywords are helpful
"Having the initial prompts is really helpful...if you don't know how to talk to [the AI], which is like half of the battle, then you're not going to get the desired result."
Adding more keywords
"If I don't have the main idea in my mind, I can just use whatever keyword is here. And you also have free typing for keywords, which is great."
Easy to use
"It's just point and click...this would be used on the therapist's laptop or desktop."
"including keywords for generating images with hands to provide better context for the AT's use"
"adding more ADLs and IADLs to expand usage scenarios, like handwriting"
"including 'build-up handle' as an option for adaptation type, as it is a widely used term in the industry"
Next...
Refining and expanding the keyword library to improve prompt creation and ensure more accurate outputs.
Conducting tests with additional OTs and individuals with disabilities to evaluate the interface and explore its potential to enhance their quality of life.
Exploring the potential of this AI tool for individuals without OT expertise who seek opportunities to access adaptive solutions.
Exploring diverse approaches to incorporating AI for OTs and individuals living with disabilities.
Optimizing the AI to specialize in designing AT adaptations.
--- back to top ---