��������

April 6, 2020

April 6, 2020

It’s been about 15 years since Matthew Tesch, class of ’07, started down the path that would lead to StaffPad, an advanced music notation app designed for composers that turns hand-written musical notation into typeset scores.

The idea took hold when Matt was an undergraduate at ��������, where he was a devoted member of the . In addition to playing bass trombone, Matt spent hours upon hours transcribing and arranging published scores, entering music into a notation program so that the orchestra could access and play them.

The process was cumbersome. “Initially, I just wanted to get music into the computer faster,” says Matt. “But the more I got interested in the technical challenge, the more I got interested in the composition experience.”

The art of musical notation

Matt was drawn to the natural flow of composing, and intrigued by how it fit into a technological world. “People often like to compose longhand, as opposed to using a computer, and the more complex a technological interface is, the more it gets in the way of the creative process,” he says. He needed to find a musical process, not a technological process, that let composers experiment with ideas and focus on the music itself.

Matt started playing with how to make writing and sharing music easier while still feeling natural. He looked into ways to scan and recognize existing music. It turned out that was easier said than done. During his senior year, he spent a semester working on it in a self-study program, but didn’t get too far. “I started to understand how complex a problem it is,” he says.

When it comes to handwriting recognition, computers are really good at breaking down and identifying symbols separated by spaces. “But with music, it’s tricky because instead of one dimension, there are two full dimensions—up and down and right and left on thepage—and it all takes on different meanings,” he says. Rather than interpreting 26 letters, there are at least as many symbols that mean different things based on where they are, making it much more complex in terms of how they fit together.

A long road

Matt’s effort became a true labor of love. After ��������, Matt went to graduate school at Carnegie Melon to study robotics and machine learning. He founded a robotics company with other graduate students before joining  as a software engineer. All the while, he worked at night and on weekends to make his pet project a reality. Then, in 2012, Matt teamed up with professional composer, arranger, and music producer David William Hearn. Their partnership sparked and the project gained traction as Matt paired his coding skills with Hearn’s musical expertise. Over the next few years, Matt shaped the app’s core technology and led the development team through the enormously complex codebase and development roadmap.

“It started out with figuring out how to take a take handwritten notation using a stylus and turn it into music,” says Matt. “But there was more to creating the app than I first thought; like, how to make an entire program with playback, undo and redo functionality—the features you take for granted but that make it viable.”

The small team persevered and StaffPad launched in 2015, enabling users to write music in their own handwriting with a stylus and edit it at any point with the convenience of touch. Users can then hear the score performed using a library of more than 55 instruments.

The app is suitable for professional composers, orchestrators and arrangers, but simple enough for the casual musician who might be intimidated by the complexity of notation—potentially inspiring them to seriously take up the art of writing music. Since its launch, StaffPad has been used in thousands of projects, from Hollywood movies to live events, and has also proved useful in classroom settings.

Upping the game

Toward the end of 2016, Matt and David reflected on the project. They saw the limitations of having the app only available on Windows, and wanted to make it available on the iPad. They also set out to create a companion app that lets users share their composition with other musicians.

This past February, they launched the companion app, called StaffPad Reader. It wirelessly displays and updates each musician’s part in real time across an entire orchestra with the tap of a button, handles page turns automatically, and keeps everyone in time with a sync’d metronome. Users can synchronize their music stands, allowing for collaborative annotations; using the original StaffPad app, the piece’s composer can move a single note and it will update instantly on every connected Reader.

Matt and David have also just brought both programs to the iPad. “Every part of the app was rewritten and improved,” Matt says. It’s taken a lot of trial and error work to get to this point. To reach it, Matt has often found himself drawing on lessons he learned at ��������. “All of �������� was a foundation, from learning to build technology to learning the life skills to be able to persevere,” he says. “School is a great place to fail safely, and taking that experience with you is really useful.”

Now that the first few post-launch weeks are behind them and scatteredbugs have largely been worked out, Matt is able to take a beat and appreciate the fruits of his labor. “It’s been great to get messages from people saying that it’s such an important part of their lives,” he says. “Someday, I’ll have time to use my own app more myself.”

See the StaffPad apps, available now on  and .

---