Thursday, May 23, 2024

Robotic Glove Provides Hope after Stroke for Piano Players

For people who have suffered neurotrauma such as a stroke, everyday tasks can be highly challenging because of decreased coordination and strength in one or both upper limbs. These problems have spurred the development of robotic devices to help enhance their abilities. However, the rigid nature of these assistive devices can be problematic, especially for more complex tasks like playing a musical instrument.

First-of-a-kind Robotic Glove

A first-of-its-kind robotic glove is lending a “hand” and providing hope to piano players who have suffered a disabling stroke. Developed by Florida Atlantic University’s College of Engineering and Computer Science researchers, the soft robotic hand exoskeleton uses artificial intelligence to improve hand dexterity.

Combining flexible tactile sensors, soft actuators and AI, this robotic glove is the first to “feel” the difference between correct and incorrect versions of the same song and to combine these features into a single-hand exoskeleton.

Robotic Glove Provides Hope after Stroke for Piano Players

“Playing the piano requires complex and highly skilled movements, and relearning tasks involves the restoration and retraining of specific movements or skills,” said Erik Engeberg, Ph.D., senior author, a professor in FAU’s Department of Ocean and Mechanical Engineering within the College of Engineering and Computer Science, and a member of the FAU Center for Complex Systems and Brain Sciences and the FAU Stiles-Nicholson Brain Institute. “Our robotic glove is composed of soft, flexible materials and sensors that provide gentle support and assistance to individuals to relearn and regain their motor abilities.”

Research Details

Researchers integrated special sensor arrays into each fingertip of the robotic glove. Unlike prior exoskeletons, this new technology provides precise force and guidance in recovering the fine finger movements required for piano playing. By monitoring and responding to users’ movements, the robotic glove offers real-time feedback and adjustments, making it easier for them to grasp the correct movement techniques.

To demonstrate the robotic glove’s capabilities, researchers programmed it to feel the difference between correct and incorrect versions of the well-known tune, “Mary Had a Little Lamb,” played on the piano. To introduce variations in the performance, they created a pool of 12 different types of errors that could occur at the beginning or end of a note, or due to timing errors that were either premature or delayed, and that persisted for 0.1, 0.2 or 0.3 seconds. Ten different song variations consisted of three groups of three variations each, plus the correct song played with no errors.

To classify the song variations, Random Forest (RF), K-Nearest Neighbor (KNN) and Artificial Neural Network (ANN) algorithms were trained with data from the tactile sensors in the fingertips. Feeling the differences between correct and incorrect versions of the song was done with the robotic glove independently and while worn by a person. The accuracy of these algorithms was compared to classify the correct and incorrect song variations with and without the human subject.