Mounya Elhilali has developed a computer model
that mirrors how humans can discern between
the sounds of a violin and a cello.
(Photo by Will Kirk)
How do you tell the difference between the sweet notes of a piano and the vibrant swell of a violin? Or, on a walk through the woods, whether the bucolic sounds are a moving breeze, swaying trees, or a flowing brook? Our brains have the ability to recognize the nature of sound, or timbre, when pitch, loudness, and other elements of sound are the same. But until now, researchers have had little understanding of how the brain processes timbre.
“We listen to a piano or violin, and our brains tell the difference immediately. But how do we build a system to do that?” wondered Mounya Elhilali, an assistant professor in the Department of Electrical and Computer Engineering at the Whiting School. The answer has begun to emerge with the results of a recent study led by Elhilali, which examines the biological bases of musical timbre perception. The study was published in November’s PLOS Computational Biology, with support from the National Science Foundation, Air Force Office of Scientific Research, National Institutes of Health, and the Office of Naval Research.
The basic idea of the study, Elhilali explains, was to develop a mathematical model that would simulate how the brain works when sound comes in, what specific information gets extracted from these sounds, and how the brain maps the sounds into different formats more suitable for processing and that allow the brain to discern these different qualities.
Basing their work on experiments in both animals and humans, they devised a computer model that can accurately mimic how specific brain regions process sounds as they enter our ears and get transformed into brain signals that allow us to recognize the type of sounds we are listening to. The model was able to correctly identify which instrument is playing (violin or piano) to an accuracy rate of 98.7 percent. The computer model was also able to mirror how human listeners make judgment calls regarding timbre, discerning more subtle differences between instruments such as a violin and cello, for example.
“Our research has direct relevance to the kinds of perceptual experiences you want to be able to give people with hearing impairments,” Elhilali says. “People with hearing aids or cochlear implants don’t really enjoy music, and part of it is that a lot of the little details are being thrown away by hearing prosthetics. [Our] results have implications for how to come up with improved sound processing strategies and how to design better hearing prosthetics so they don’t discard a lot of relevant information.”
She hopes to advance the research to understand how a system could differentiate among sounds such as human voices or machine guns or a siren or chirping birds. In addition to improving the design of hearing prosthetics, practical applications include computer systems that can annotate musical multimedia data or transcribe musical performances.
“My interest in this line of work really came from my fascination with the brain, figuring out how it works and what makes us the beings that we are,” she says. “The ultimate goal of my research is to build systems that can borrow from principles of brain function in order to design better sound technologies that can hear, interact, and learn from their environment, and autonomously make decisions.”