The quest to create a camera that can see the way we see—continuously, selectively, alert to movement and meaning—is the challenge facing Ralph Etienne-Cummings, chair of the Department of Electrical and Computer Engineering and an expert on human senses and the digital processes that one day may replicate or even exceed them.
Etienne-Cummings is a specialist in the field of neuromorphic engineering— the attempt to construct electronic signal-processing systems that are inspired by the neural circuits inside a living human or animal brain.
Working with research associates at the Vision Institute in Paris, Etienne-Cummings is designing a machine-vision system that is able to activate individual pixels only when their infinitesimal field of view receives new information, just as our eyes ignore a static background until something occurs there—a fly on a wall or a flash of lightning outside a window—to divert our attention.
“Standard cameras cannot do this,” he says. “Basically, you have a single shutter that opens and closes, and then you have to get the information out. The information you get is the average over the time that the shutter was open. But when the shutter is closed, you get no information at all.”
Even though exposure times have decreased from hours at the dawn of the daguerreotype in the 1840s to milliseconds in our smartphone cameras today, the frame-by-frame basis of photography remains an obstacle.
The goal of neuromorphic engineering, Etienne-Cummings explains, is to bring “always-on”—yet low-power, battery-conserving—capability to cameras in robots and medical devices for such roles as 3-D mapping and quick response to movements within the visual field. Such devices would be capable of interpreting the real-time streams of continuous data that our eyes and brain handle without conscious effort.
“To fully unlock the potential of eyelike vision sensors, you need to abandon the whole notion of a video frame,” Etienne-Cummings writes in the IEEE Spectrum. “That can be a little hard to get your head around, but as soon as you do that, you become liberated.”