When Arun Nair began his summer internship at Snapchat in New York City, he knew it would be research heavy and that he would be joining a team designing new products under Snapchat’s umbrella.
However, Nair did not expect to have a hand in the creation of technology so interesting that he would write a paper about it (“Audio Visual Zooming: What You See is What You Hear”). The paper was accepted at the ACM Multimedia Conference, which took place in late October in Nice, France, and won the Best Paper Award.
“I expected to be challenged, but I didn’t think it would lead to a Best Paper Award,” Nair said. “I learned so much from this experience and had a lot of fun throughout the process.”
Nair is a PhD candidate in the Department of Electrical and Computer Engineering who is co-advised by Trac Tran, professor and Muyinatu Bell, assistant professor. Austin Reiter, an assistant research professor who has an affiliation with the Department of Computer Science at Johns Hopkins, supervised Nair at Snapchat, along with Changxi Zheng and Shree Nayar, faculty at Columbia University.
Nair’s work at Snapchat looked at specifying audio capture while recording video. If someone wants to record a video on their cellphone, they can zoom in and focus only on the subject matter they want to emphasize.
However, there is no similar option that allows people to record only the audio they want, cancelling out unwanted sounds occurring in range. The goal was to create technology that enabled that.
“To solve this problem we used beamforming, a technique that is also used in ultrasound imaging, which is the work I do here with Dr. Bell. We developed a new algorithm that mathematically formulated this issue as a zooming problem, and linked the visual field of view to the audio field of view,” Nair said. “It was quite different from the work the conference is used to receiving, which usually focuses on vision with audio secondary. We did the opposite with the innovation being on the audio side of things, and it also helps that this is really useful technology that everyone can appreciate.”
Nair believes the whole experience with Snapchat has helped him grow tremendously.
“The team at Snapchat was great, constantly providing constructive criticism and support. That helped me grow as a researcher, and I also discovered that I enjoy the unique challenge of the research internship setting,” Nair said. “The coolest thing my experience taught me is that it is possible to find and solve really interesting problems in elegant ways that bridge the academia-industry divide, which I want to continue to do. Lastly, there was a lot of emphasis placed on developing my ability to effectively present research, which has definitely been a help.”