The College of Design congratulates Ning Yang for the successful defense of their dissertation, "Mechatronics driven design approach for musical expressivity in robotic musicianship" on Monday, December 12th, 2022.

Abstract:

Mechatronics driven design approach for musical expressivity in robotic musicianship

In this dissertation, we present mechatronics-driven design approaches that allow robotic musicians to perform with musical expressivity. We introduce three research projects in developing a robotic marimba player, a wearable drumming prosthesis, and a robotic guitarist.

The first project, the development of a robotic marimba player, focused on the design of a novel striking system with brushless direct current motors. The new striking system allows the marimba robot to achieve a wider dynamic range, faster speed, and more complex marimba playing techniques. We objectively evaluated the dynamic range, speed, and four marimba techniques of the new striking system, showing that the new system has a wider dynamic range, a higher speed, and more capable techniques than the old striking system. We conducted a listening test to show that the robot was able to achieve human-level expressivity. We also conducted a Turing test to show that the subjects could not differentiate the robot from human players playing four marimba techniques.

The second project shifted to a wearable drumming prosthesis. The purpose of this wearable robotic device is to help an amputated drummer to regain his grip and control of drumsticks. We introduced a quasi-passive robotic drumming prosthesis that reads the drummer’s grip from forearm muscles through electromyography sensors and manipulates the drumsticks’ dynamic behavior with corresponding stiffness parameters. We demonstrated that the drummer was able to use switch grips and change natural bouncing patterns among single, double, and triple rolls. We also validated that the drummer was able to utilize the robotic prosthesis to perform advanced drumming techniques, such as paradiddle.

The last research project focused on designing a new robotic guitarist. We analyzed and modeled how humans played the guitar and created an expressive robotic guitarist with a right-arm module for strumming and picking and a left-hand apparatus for pressing the strings. We evaluated the dynamic range, speed, microtiming control, noise level, and guitar techniques on this new robotic platform. Through the subjective listening test, we also found that the robotic guitarist was able to perform at human-level expressivity.  

Through the development of these projects, we have demonstrated that, with human-modulating driven mechatronics design, we can significantly improve the physical capabilities of robotic musicians, supporting more musical expression through a wider dynamic range, a more subtle microtiming control, and a more extensive range of music playing techniques.