Integrate-and-Fire Array Transceiver for Image Processing

Project Personnel

Jamal Molin, John Rattray, Chetan Singh Thakur


Within the field of neuromorphic engineering, we seek to design systems with facsimiles to the mechanisms and communication protocol of biological neurons. By doing so, we can overcome the challenges of designing systems which are low-power and low-area while maintaining computational efficiency and accuracy. Naturally, these neuromorphic systems can be used for directly modeling various tasks exhibited by neurons in the brain (i.e. neural networks).


The brain performs complex computations by operating a large number of relatively simple neurons in parallel, with a high degree of interconnectivity between cells. Because many features of this processing power emerge from the network architecture, it is useful to generate computational models of potential configurations to explain experimental observations and stimulate future work. Traditionally, these simulations are generated in software and executed over the course of a few hours. However, if we are to build autonomous devices that emulate the operation of the brain, our circuits must function in real-time. Additionally, we are interested in creating an interface between the simulations and biology to evaluate their function in situ, and this will also require real-time operation.

Integrate-and-Fire Array Transceiver


Previously, we designed and implemented a reconfigurable hardware network (the IFAT) of 2,400 neurons that is capable of operating in real-time. A mixed-signal design, the IFAT (integrate-and-fire array transceiver) performs analog computations internally, while communicating with the external world in the digital domain. To allow unlimited connectivity and reprogrammability, there are no hardwired connections between neurons. Instead, network topologies and synaptic parameters are stored in a look-up table in off-chip RAM, and a microcontroller is responsible for routing action potentials to their appropriate targets. To achieve high throughput and ensure interoperability with other neuromorphic chips, the digital communication protocol known as address-event representation (AER) is used both between neurons in the IFAT and between the IFAT and external hardware.

Mihalas-Niebur Neuron Model for IFAT


Currently, we are working on completing an IFAT chip that uses the Mihalas-Niebur neuron model opposed to a leaky integrate-and-fire  neuron model.  The Mihalas-Niebur neuron model is a more generalized model which consists of an adaptive threshold.  This model allows for generating 20 of the prominent neuron spiking behaviors.  The neurons on the IFAT chip will be based on this model and we seek to do this using a low-power, small-size approach allowing for a dense array of neurons while consuming minimal energy per spike/event.  Preliminary results of this work was recently published at BioCAS 2015 (V. Varghese et. al, 2015).

IFAT for Real-time, Lower-power, Image Processing for Aerial Vehicles


We are also interested in utilizing the IFAT system for performing stochastic, event-based image processing in real-time under low-power and small-size constraints.  The application of interest is for micro, unmanned aerial vehicles.  Most hardware-based image processing systems are built on von Neumman-type architecture.  Digital, fixed-point arithmetic is used to perform various computations.  However, such computation is highly susceptible to error when noise is present.  As technology scales down and supply voltages are lowered, it is becoming more and more important for our systems to be robust to noise.  Henceforth, a neuromorphic approach that does computation and represents information via events/spikes over time is a much more reliable approach; it is more robust to noise, consumes less-power, and better mimics functionality of neurons in the brain.  In our work we seek to use the IFAT for doing various image processing tasks such as dewarping, filtering, tracking, etc. in real-time while coupled to an aerial vehicle.  Preliminary work has been published in both MWSCAS 2015 and ISCAS 2016 (J. L. Molin et. al, 2015, J. L. Molin et. al, 2016).

Continuing work includes coupling this IFAT system with an event-based camera called the ATIS (Asynchronous Time-based Image Sensor).  The ATIS camera is not a traditional frame-based camera but instead outputs events only when their is a log intensity change in illumination above a set threshold.  A change event followed by two successive events triggered by a low- and high- threshold are used to determine pixel illumination at the pixel location where the change occurred.  Intensity is encoded in the time between these two successive events opposed to traditional fixed point, digital output.  And AER communication protocol is used for the ATIS making it ideal for our event-based IFAT system.  We seek to use this camera as input to our IFAT system for performing various image processing tasks in real-time.

Recent Publications

  • Andreou, Andreas G., Tomas Figliolia, Kayode Sanni, Thomas S. Murray, Gaspar Tognetti, Daniel R. Mendat, Jamal L. Molin et al. “Bio-inspired system architecture for energy efficient, BIGDATA computing with application to wide area motion imagery.” In 2016 IEEE 7th Latin American Symposium on Circuits & Systems (LASCAS), pp. 1-6. IEEE, 2016.
  • J. L. Molin, J. Rattray, R. Etienne -Cummings, “Stochastic Image Processing and Simultaneous Dewarping for Aerial Vehicles”, ISCAS, 2016.
  • V. Varghese, J. L. Molin, C. Brandli, S. Chen, R. Etienne-Cummings, “Dynamically Reconfigurable Silicon Array of Generalized Integrate-and-Fire Neurons”, BioCAS, 2015.
  • J. L. Molin, T. Figliolia, K. Sanni, I. Doxas, A. Andreou, and R. Etienne-Cummings, “FPGA emulation of a spike-based, stochastic system for realtime
    image dewarping,” Midwest Symposium on Circuits and Systems, 2015.