ASU faculty awarded patent for shoulder rest that lets violinists feel the sound

For years, violinist Seth Thorn, an instructor in Arizona State University's School of Arts, Media and Engineering, wanted to expand how traditional instruments were played, learned and used to perform.

“I’ve been tooling around with my acoustic violin and digital technology, using a computer to transform the way it sounds when you play it,” Thorn said, “but there’s always been something disjointed about feeling this vibrating, sounding instrument under your neck that’s completely unrelated to the transformed sound you’re hearing from the computer.”

Thorn said digital musical instruments often lack elements of traditional musical instruments that “make them so rich to play and to learn, like having sound come out of the body of the instrument rather than remote speakers” or feeling sonic vibrations in the tips of your fingers when you play. So Thorn decided to change that. 

“Lots of violinists use a shoulder rest to ease strain on their neck when playing,” he said. “When I realized I could add some small transducers to it to make you actually feel the digital sound you’re hearing, it completely transformed my practice — a real ‘eureka’ moment for me that has brought lots of joyful playing of this hybrid instrument.”

Thorn and Clinical Assistant Professor Byron Lahey designed the new haptic feedback violin shoulder rest and have been awarded a U.S. patent. This digital extension for the violin can be used to replace the standard shoulder rest on violins for ergonomic support. Unlike the traditional shoulder rest, this digital shoulder rest is designed to serve as a silent metronome, to assist hearing disabilities and to entrain rhythm, all by using haptic feedback as its delivery method.

ASU Skysong Innovations is helping to market the device to a variety of sound and music industry parties.

WATCH: View a full demonstration of the hybrid-feedback shoulder rest.

Thorn answered some questions about this new digital extension for the violin and what inspired him to create the device.  

Question: Has any of your past research influenced this project? If so, how and what projects?

Answer: My research relates movement and computing through the lens of the violin. How can we track the salient gestures and movements in violin playing with sensing technology and couple them to sound processing to give the feeling that your violin is a very different instrument? The violin is already so incredibly expressive, so this is a big challenge. Extremely subtle changes in the tension of your wrist and fingers yield very different expressive dynamics, which is why it takes 10, 20 or 30 years to learn how to play it expressively. So you might think, what’s the point of trying to “augment” the violin with digital technology if it’s already so refined? Well, it’s really this refinement and nuance that has so much potential impact for human-computer interaction research: If you can be attentive to how these microscopic changes inform the sound production, then you’re on the way to learning how to design highly refined digital interfaces and responsive media more generally. Musicians and artists are experts in this — we need more of them in this kind of research! 

Q: How are you hoping this haptic shoulder rest will impact those learning violin?

A: There’s this feeling people have, I think, that the violin is “complete,” that it’s reached the apex of its development and there’s nothing left to do. But it’s important to remember that the violin has had five centuries to develop, and that along the way, decisions were made that gave the violin a definite character, like the tension of the strings, the shape of the bow, etc. But these decisions could have been made otherwise. The big philosopher of technical evolution, Gilbert Simondon, reminds us that even the most “concrete” technical objects are still “abstract,” that is, there’s always more potential in these objects that can be teased out by experiment. Putting haptic feedback technology into a violin shoulder rest is one example: Nobody ever anticipated this, but the potential has been there all along.

Now, to get to your question — what if the practice of violin playing evolved into a practice where you not only learn the instrument, but you also change it, creatively transforming it using digital technology with an app on your phone? People learn, create and are motivated in so many different ways, and maybe some people would want to make this part of their practice, even the center of their practice. It’s a big motivation. Let’s see what happens in the next 500 years.

Q: How is Skysong partnered with this project?  

A: I showed the shoulder rest to a colleague at the ASU Synthesis Center who’s familiar with my work, and his immediate response was, “You should patent it!” I was so encouraged by the enthusiasm, but I also had a lot of uncertainty about what the process might entail, and I felt unprepared for it. Then I learned about Skysong. They took care of the piles of paperwork related to the utility patent, and now we’re discussing licensing the technology to some big industry partners in music technology, or even creating a startup. Skysong has been great — they take care of the business and red tape stuff, and I get to keep my creative focus on developing and improving the technology. 

Q: Can this hybrid technology positively affect a certain group of people or demographics?

A: Haptic technology introduces a lot of exciting possibilities for people with sensory impairments or learning difficulties. In particular, you can improve sound or music perception for individuals with hearing loss by adding haptic feedback to increase the sensory dimensionality of a digital interface — or even a violin! With the violin, in fact, very little haptic feedback comes from the base of the violin into the neck and collarbone of the player. Most of the information you get is from the mechanoreceptors in your fingertips, which are most sensitive to vibrations around 250 Hertz. So adding haptic feedback felt by the collarbone and neck, delivered by this haptic technology, doesn’t disrupt those natural dynamics. It just adds feedback to a felicitous space. 

Q: How will this technology benefit or change live performances? 

A: If you’re playing in a rock band, you get your own amplifier next to you in order to localize your sound. It makes you hear and feel that the sound you’re contributing with your instrument is emanating from your position on the stage. My big vision with the haptic violin technology is an ensemble of violinists who, in the same way, confidently play their digitally transformed instruments with the help of this local, multisensory integration provided by the shoulder rest.

There are also opportunities for radical new ways to compose music. Like, instead of having a traditional score and conductor coordinating an ensemble, what if the “score” consisted in “taming” a capricious instrument, by playing it, that seems to have a life of its own. The haptic elements in my shoulder rest are basically small speakers, and if you drive them hard enough you can make the body of the violin resonate in really captivating ways. There’s an opportunity here to rethink the way ensembles coordinate, and how we can play together differently, or even what it means to play “together.” We can use haptic signals to cue that. There’s really a lot of versatility with this technology.

Q: What are your long term goals and plans for this project?

A: So to get to the big picture — the really big picture here — I’m going to refer to Jacques Attali, a famous French economist and finance minister who wrote an influential book in 1985 called “Noise: The Political Economy of Music,” in which he predicts and celebrates the return of “very ancient forms of production,” and music for “immediate enjoyment” rather than “spectacle.” The haptic shoulder rest really isn’t anything until you couple it with digital technology that transforms the sound of the violin in some way, so the question becomes: How do you do this? How do artistic decisions about the response actually evolve and come to be? You code some stuff, then you pick up your violin and give it a try. Then you make some adjustments, or you throw everything out and try something new, and so on.

By experiment, you negotiate the action of playing the violin with the multisensory perceptual feedback you get. And you learn a lot as you do this. Now, when you really think about it, this is an extraordinary scene, because it’s precisely that “ancient mode of production” Attali is talking about, even though it’s mediated by the most advanced real-time computing capabilities we have nowadays. It’s like crafting a violin, but instead of finding the right wood, carving it, “tap toning” it, reshaping it and so on, you’re doing all of this with code. But it’s still artisanal work, not industrial work — it’s very highly refined. Adding haptic feedback enriches the whole process.

So, to answer your question, my big vision for this project is the proliferation of these personal musical instruments that recapitulate ancient artisanal practices by means of advanced digital signal processing — a felicitous merging of bits and atoms. The haptic shoulder rest gets you closer. I don’t want to forget to mention my colleague Byron Lahey, who helped build the first prototype with me and shares part of the patent. While I could've done this project on my own, it would’ve been a big mistake not to solicit his advice and abilities as a master craftsman. We share an office in Stauffer B, so the collaboration just emerged organically. This is one of my favorite things about AMESchool of Arts, Media and Engineering: All these people of very different abilities, with collaborative instincts, in a big shared space. I also want to thank violinist Katherine McLin of the School of Music. She’s such a prolific and talented musician, but also very forward thinking. She brought her studio over to the Synthesis Center to try out my hybrid violin. The whole event was a lot of fun.