
(image: Thumb Piano app)
Having been in a situation where I lost control of my fine motor skills, including use of my fingers, I wanted to create a musical instrument that can be played by those with weaker upper body strength. The way it works now is that when the mouse is rolled over each key, it emits the corresponding sound.
I have also included facial gesture as a mode of input. Therefore, when a user looks at the thumb piano they can play it with their face.
Mouse Over Version
Facial Gesture version (uses the web camera)
The process:
The device was made using p5.js, a tool for creative coding. I recorded the actual notes from playing the tines of the thumb piano, and mapped them to each key.
I also used the ML5 gesture recognition model, and mapped it to the thumb piano keys.
User Paths
User 1
- Parent or Caregiver that wants to teach African instruments to a child.
- Willing to use a kit or instructable
- Willing to download the app for the sound synthesis connected to this device
User 2
- A music lover that is into experimentation
- Is able to tinker with low-energy electronics
- looking for cultural references in music
- Wants to connect to African Culture
User 3
- An occupational therapist looking for tools to help with dexterity
- Is willing to make the kit with a patient
- Believes in music therapy
- Buys into the whole instructables idea
User 4
- Someone wanting to use the app only
- Wants to also mess around with gesture-controlled devices
- The device and the app instrument are the same object
User 5
- A person with limited mobility
- Looking to play music but does not have the muscle strength (I say this from personal experience)
- Enjoys music therapy
- Willing to connect to African culture
User Mapping
Idea |
Metaphor |
Model |
Display |
Play Music without the use of hands for arm mobility-challenged people. |
![]() Moving to music |
![]() Use ML5's gesture control model, with p5.js piano keys example |
![]() Kalimba app interface |
Error |
Scenario |
Task |
Control |
![]() Some people have weak arm / hand muscles and can not play a traditional thumb piano. |
![]() The app is used for therapeutic purposes, to engage the senses. perhaps in occupational therapy. |
|
A face tracking app to play the sounds of a traditional thumb-piano.
|
Above: Verplank's IxD framework
