^{1}

^{2}

^{1}

^{2}

Edited by: Hava T. Siegelmann, Rutgers University, USA

Reviewed by: Alexander G. Dimitrov, Washington State University Vancouver, USA; Jonathan Shaw, Sensory Inc., USA

*Correspondence: Dana H. Ballard,Department of Computer Sciences, University of Texas at Austin, Austin, TX 78712, USA. e-mail:

This is an open-access article subject to a non-exclusive license between the authors and Frontiers Media SA, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and other Frontiers conditions are complied with.

A prominent feature of signaling in cortical neurons is that of randomness in the action potential. The output of a typical pyramidal cell can be well fit with a Poisson model, and variations in the Poisson rate repeatedly have been shown to be correlated with stimuli. However while the rate provides a very useful characterization of neural spike data, it may not be the most fundamental description of the signaling code. Recent data showing γ frequency range multi-cell action potential correlations, together with spike timing dependent plasticity, are spurring a re-examination of the classical model, since precise timing codes imply that the generation of spikes is essentially deterministic. Could the observed Poisson randomness and timing determinism reflect two separate modes of communication, or do they somehow derive from a single process? We investigate in a timing-based model whether the apparent incompatibility between these probabilistic and deterministic observations may be resolved by examining how spikes could be used in the underlying neural circuits. The crucial component of this model draws on dual roles for spike signaling. In learning receptive fields from ensembles of inputs, spikes need to behave probabilistically, whereas for fast signaling of individual stimuli, the spikes need to behave deterministically. Our simulations show that this combination is possible if deterministic signals using γ latency coding are probabilistically routed through different members of a cortical cell population at different times. This model exhibits standard features characteristic of Poisson models such as orientation tuning and exponential interval histograms. In addition, it makes testable predictions that follow from the γ latency coding.

Although individual visual cortical neurons exhibit deterministic spiking in special instances, e.g., during sleep cycles (Drestexhe and Sejnowski,

Specialized interpretations of the spike code have been made, for example that action potentials literally signal Bayesian statistics (Ma et al.,

A general way that cortical neurons can be characterized is in terms of their receptive fields. Collections of these can be interpreted mathematically as a library of functions, termed basis functions, that can encode any signal in terms of a sum of pairwise products of the basis functions and associated coefficients. Thus, a vector of sensory or motor data values distributed over a space

A basis function specified by the vector _{i}_{i}

Tremendous progress has been made in developing mathematical formulations that solve both of these problems in a way that explains biological features, such as the orientation distributions of simple cells in striate cortex, by adding a term to Eq.

The sparse coding results are important, but so far the majority of models have used abstractions of neurons with signed real numbers for outputs, which in turn assume some kind of population coding, instead of directly dealing with action potential spikes. The crucial issue is: if the cortex is to use a minimal number of spiking cells, how are they to communicate analog values? There have been several ingenious methods to do so by exploiting timing codes (Eliasmith and Anderson,

One possible timing resource is the cortex's rhythmic signals in the γ range (30–80 Hz). It has been suggested that the cortex might use an analog latency with respect to the phase of a γ frequency oscillatory signal to do this (Buzsáki and Chrobak,

Another important consequence of a latency code is that it is compatible with spike timing dependent plasticity (STDP; Bi and Poo,

We have proposed a specific model based on γ latencies (Jehee and Ballard,

The key testable predictions of the model concern properties of the latency coding of spikes. If the stimulus is represented by a latency code, many repeated applications of the same stimulus, on average, should result in the reuse of the same latencies. Thus, the distribution of latencies in any neuron, measured with respect to the γ timing phase, should be highly non-uniform. This prediction is in direct contrast to that of a Poisson model, which would predict a uniform distribution that is independent of any particular γ reference. A subtler prediction involves the fundamental constraint between the spikes of neurons with overlapping receptive fields. If only one spike from the overlapping group is sent, then the groups’ spikes should not be correlated at short (1∼3 ms) timescales. If more than one spike is sent during that period, the component latencies would have to have been adjusted accordingly. Neurons with non-overlapping receptive fields can be correlated, and in fact should be, as the representation of stimuli is still distributed across multiple cells, albeit in much smaller groups than needed by rate-code models.

The development of neural receptive fields in the striate cortex has received extensive study and has become a standard venue for testing different neural models. Thus, this circuitry provides an ideal demonstration site for our very different model of cortical neuron coding and signaling. The more abstract version of the algorithm has been shown to learn receptive fields in cortical areas V1 and MST (Jehee et al.,

What makes the model fundamentally different is its combination of three interlocking constraints: (1) randomized action potential selection, (2) variable γ latency coding, and (3) multiplexing of several different γ range frequencies.

A central constraint on action potential generation concerns neurons with overlapping receptive fields. Two receptive fields are said to overlap when the dot product of their normalized receptive fields is significant, which we take to be greater than some scaler value μ. In our simulations most neurons are nearly orthogonal, that is, the dot product is less than 0.20. Rather than selecting the most similar neuron at each instant, neurons with significantly overlapping receptive fields compete to be chosen (Jehee et al., ^{10r}/

_{2} to _{1}. In the case of multiple overlapping receptive fields, the odds are distributed appropriately among them.

A general way that all cells can signal coefficient information is via γ latencies, as shown in Figure

where

The assumption of the ubiquitous use of latency coding is that it allows numerical data to be propagated throughout the relevant cortical circuitry quickly and, at the same time, owing to the sparseness of the code, makes this circuitry relatively insensitive to cross-talk from any other spike traffic. Given a set of learned receptive fields, an input stimulus initially can be represented by selecting a candidate probabilistically from amongst the set of V1 neurons that have receptive fields that are similar to the input, subtracting the candidate's receptive field from the input, and repeating this process with the resultant residual acting as a new input. The specific algorithm to do this has been described in (Jehee et al.,

where _{n}

A key assumption in the model concerns the mechanism for realizing this equation in neural circuitry. The use of γ latency coding to represent quantities suggests that the process of evaluating residuals should be done in one or two γ cycles. Otherwise it would take too long to represent the input stimulus. For example, since each cycle consumes on the order of 20 ms, if it took a cycle for each of 10 coefficients, the total time budget would be unrealistic. Thus, our model posits that an essential role of lateral connections in V1 is to implement the subtraction process shown in Eq.

Another important constraint is that, in order to be latency coded without error, the residual magnitudes need to be generated in decreasing order. This is usually the case as highly overcomplete representations mean that some neuron's receptive field is close to the residual and thus has a high probability of being chosen. This probability can also be tuned by scaling β in Eq.

Learning the receptive fields starting from random connections is straightforward. At each step, the winning neuron has its receptive field made a little more similar to that of the stimulus that resulted in its selection. This can be done by moving each receptive field in the direction of its residual, i.e., the

where

This equation shows that the learning of receptive fields can take place simply using STDP if both use the same latency code. Since (1) Δ

The γ band is a broad range and consequently it would be unlikely to expect a single frequency to be chosen and used as a “clock.” Indeed elaborate statistical tests for such a clock in local cortical field potential data over 2–4 s have been negative (Burns et al.,

where γ_{o}

The focus of our simulations is to show that timed circuits that use the randomized spike generation protocol send information in a very different way than is currently conceived in that the group of cells sending the information varies from cycle to cycle. However, despite this difference, the spikes in this code can appear to be very similar to those generated by a Poisson processes. At the same time, the latency code introduces regularities in the spike distributions with respect to the γ phase that should be detectable.

Does this process generate data that describes conventional receptive fields? We tested the claim that rival neurons were competing in simulation. Neurons whose receptive fields were learned with the matching pursuit algorithm that used the conventions of our model were tested on an 8 × 8 simulation by using small Gabor image patches as input. The Gabor patches were computed using the parameters (wavelength, phase offset, bandwidth, aspect ratio) = (2, 0, 1, 1). The wavelength is measured in pixels.

Thirty-six Gabor image patches were created, one for every 10° rotation. These were then presented to the network 1200 times and fit with learned basis functions each time. To emphasize the point that the randomized neural selection process models the receptive field, we chose two basis functions that overlap and measured their receptive fields using their spike counts for the different Gabor orientations. Their histogram data are indicated in Figure

The Gabor image test shows that a probabilistic selection method can produce orientation tuning but does not show off two crucial features of the coding method, namely (1) the codings of an image patch vary from γ cycle to γ cycle and (2) they use latency coding to send a coefficient. These two features are made explicit in Figure

The simulation uses 256 neurons and shows the spikes for each of 200 γ cycles, representing 4 s of elapsed time (200 × 20 ms). Each colored dot represents a spike for a particular neuron and in each column 12 such spikes are present, 1 from each of 12 neurons selected. The spikes are color-coded to indicate their γ latency values, with light yellow to white being the highest value and dark red being the lowest. The colored scale bar indicates the values of the coefficients prior to latency conversion. Neurons that are used frequently have high coefficient values.

The 10 inset images in a row at the top of this figure are reconstructed from the sum of products of the 12 coefficients at cycles {20, 40, 60, 80, 100, 120, 140, 160, 180, 200} and these images show a high degree of invariance despite the fact that the basis functions used in encoding the patch are different. For example, comparing the basis functions for cycles 60 and 120 (shown on the extreme right hand side) shows that the neurons used to represent the patch are almost entirely different. This can be checked visually by picking a receptive field for the 60 ms patch encoding and trying to find its counterpart in the 120 ms patch encoding and

One direct consequence of the probabilistic spike selection strategy is that if the individual neurons are examined, their spike “rate” is correlated with their average response coefficients. Here we use “latency code” as a synonym for “response” since a recipient neuron can decode the latencies to recover the responses. To demonstrate this feature, Figure

The data in Figures

In the first test, we constructed a more complete spike data set by embedding those spikes in a “noise bath” of background neural spiking. The idea of noise here is as a model for any other ongoing processing. For each neuron, the probability of firing at each millisecond was set to 0.008, representing approximately 10 Hz background rate, and the model's spikes were added to this bath. For comparison, we compared these spike trains to a pure noise bath with firing probability of 0.01. The different background rates were chosen so that the number of spikes in each case was approximately the same. The results are shown in Figure

The test shown in Figure

An issue that has been finessed up to this point is that γ frequencies appear in a range of 30–80 Hz. Thus per (Ray and Maunsell,

Figure

Figure

Our primary hypothesis is that the rate code observed in so many experiments might be an abstraction of a more fundamental and efficient latency code that uses frequencies in the γ range as timing references but can generate rate-like statistics when tested by conventional means. This stands in contrast to models that posit that rate codes and γ frequency codes are separate and compatible (e.g., Masuda and Aihara,

However the γ latency code is not without challenges. It is significantly more technically demanding to implement than the standard rate code and the following issues need to be addressed in future work that would use a more detailed neural model.

^{4}.

^{3} slots would be available. To reduce cross-talk, it may be possible to organize all the connections for any given code on the same branch of the dendritic tree.

When phase locked, the γ latencies may be difficult to detect. We envision that the temporal usefulness of a particular set of γ latencies might be as little as 300 ms, the typical time used to acquire information with a saccade and commensurate with the times used in visual routines (Roelfsema et al.,

The increasing number of experiments that have confirmed the general presence of the γ signal has led to many suggestions as to its role. The γ signal has been suggested as the basis for a number of effects such as attention (Womelsdorf and Fries,

The number of independent computations that are possible in the cortex is an open question that the γ signal may also shed light on. It might still be possible that the cortex is capable of performing just one large computation at a time, so that all the participant neurons implicitly reference that computation and no further bookkeeping is necessary. Nonetheless, a way to segregate different neural computations in cortex would add enormously to its computational capabilities as different tasks could be ongoing. Such tasks, as pointed out by von der Malsburg (

The huge disparity between the precision of digital computers and neurons in the brain prompted computer pioneer von Neumann (

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Assume that a temporal interval can be adjusted so that in a Poisson model, the probabilities of having zero or one spike dominate the probability of having more than one spike. In that case, an estimate of the probability of an input rate ɸ can be made by counting spikes. Where the input is a probabilistic variable

where

To just get within 10 with 95% confidence requires about 150 synapses. So for 100 independent inputs the total synaptic budget necessary would be 150 synapses per input ×100 inputs, or 15,000 synapses total. In contrast, if the latency code has 10 discernable levels the total number of inputs required would be just 100.

The learning algorithm is a fast algorithm for fitting learned receptive fields to input data based on matching pursuit (Mallat and Zhang,

The input is obtained from 768 by 768 black-and-white images of natural surroundings (Figure

where the tilde represents the Fourier transform in 2D, and _{0} = 300 cycles/image.

We limit this input into the model to a 10 by 10 (100) “patch” that is randomly selected from the filtered image, and represented as a single vector.

The model, which would correspond to an orientation column in cortical area V1, is represented by 256 units. In the language of the model, the synaptic weights between its un-oriented input and oriented V1 units form basis vectors that represent the preferred stimulus of the model oriented V1 neurons. These cells predict its layer IV input _{i}_{i}

in which

To choose the

at the next time step, an additional vector is chosen that minimizes

and so on, where the response

This deterministic version was modified in order for the learning algorithm to be optimal in terms of sparseness (Jehee et al.,

where _{k}_{j}^{−1} = 1/10 is a temperature parameter and

In the modified model, the probability with which a unit is selected increases when its receptive field structure better predicts the lower level input. To guarantee optimality, the response of a selected unit

The feedforward–feedback cycle is then repeated on the residual input so that after

In words, the number of active V1 neurons increases at each time step in the model, and their combined prediction is subtracted from the actual input. We assume model V1 responses to be stable and non-decaying over the considered time scales.

To enhance the sparseness of the neural code and better capture the input statistics, basis vectors are updated in each feedforward–feedback cycle. This is done by minimizing the description length of the joint distribution of inputs and neural responses (Jehee et al.,

where

The response of a neuron encodes an analog value in terms of a latency with respect to the phase of a γ signal. The particular formula used is given by

where

This material benefited from discussions with Fritz Sommer and Bruno Olshausen and the Redwood Center for Theoretical Neuroscience at Berkeley, as well as colleagues Ila Fiete, Jonathan Pillow, Nicholas Priebe, Dan Johnston, Bill Geisler, Larry Cormack, and Alex Huk at the the University of Texas at Austin and Jochen Triesch and Constantin Rothkopf at FIAS, Frankfurt. The research has been supported by grants MH 060624 and EY 019174.