Creating NeuroInformation Processing Machines
We devised 4 classes of NeuroInformation Processing Machines: Time Encoding Machines, Channel Identification Machines, Phase Processing Machines and Spike Processing Machines. The overriding question raised by Time Encoding Machines is "Can stimuli such as visual fields and auditory scenes be faithfully represented in the spike (time) domain"? Using a formal theorem/proof framework, we answered this question in the affirmative: if the average spike rate at the output of the Time Encoding Machine is beyond the Nyquist rate, perfect representation of the stimulus can be achieved. A duality result between Time Decoding Machines and Channel Identification Machines settles the question of identification for the latter.
Time Encoding Machines (TEMs) are asynchronous signal processors that encode analog information in the time (spike) domain. Time Decoding Machines recover the encoded stimulus from the time (spike) sequence. Using frame theory, the theory of dynamical systems, statistics and machine learning, we demonstrated for the first time that faithful recovery of natural video (movies, animation) and auditory scenes (speech, sounds) can be achieved under Nyquist-type rate conditions.
Channel Identification Machines (CIMs) identify receptive fields in circuit models that incorporate biophysical spike-generating mechanisms (e.g., the Hodgkin-Huxley neuron) and admit both continuous sensory signals and multidimensional spike trains as input stimuli. We demonstrated a fundamental duality between the identification of the dendritic stimulus processor of a single neuron and the decoding of stimuli encoded by a population of neurons with a bank of dendritic stimulus processors. We have also shown that identification algorithms can be effectively and intuitively evaluated in the stimulus space.
The early visual system, being mostly analog (graded potentials), processes phase information commonly associated with edges and moving edges arising in visual fields. Phase Processing Machines draw on a novel data structure, called generalized local phase, for extracting edges largely independent of light intensity. In addition to visual processing, generalized local phase information can be smartly employed in speech processing. Underlying phase-based motion detection is a feedforward divisive normalization circuit.
There is strong evidence showing that divisive normalization may contribute to gain control in olfaction, vertebrate retina, primary visual cortex, primary auditory cortex and sensory integration. Feedforward divisive normalization has been proposed as a model of canonical neural computation, and used in nonlinear image representation for achieving perceptual advantages. This computation is key to many sensory processing circuits underlying adaptation and attention. Feedforward normalization is also frequently used in deep neural networks
The brain must be capable of forming object representations that are invariant with respect to the large number of fluctuations occurring on the retina. These include object position, scale, pose and illumination, and the presence of clutter. Combining visual and auditory stimuli corresponding to a person talking at a cocktail party can substantially enhance the accuracy of the auditory percept. What are some plausible neural mechanisms by which invariant object representations and demixing of auditory information can be achieved? Spike Processing Machines implement these and other neural mechanisms in the spike domain.