^{1}

^{2}

^{3}

^{†}

^{3}

^{*}

^{1}

^{2}

^{3}

Edited by: Benjamin R. Arenkiel, Baylor College of Medicine, USA

Reviewed by: James Schummers, Max Planck Florida Institute, USA; David J. Margolis, Rutgers University, USA

*Correspondence: Stephen D. Van Hooser, Department of Biology, Brandeis University, 415 South St. MS008, Waltham, MA, USA e-mail:

This article was submitted to the journal Frontiers in Neural Circuits.

†Present address: Marisa Kager, Harvey Mudd College, Claremont, USA

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

Neurons in the visual cortex of all examined mammals exhibit orientation or direction tuning. New imaging techniques are allowing the circuit mechanisms underlying orientation and direction selectivity to be studied with clarity that was not possible a decade ago. However, these new techniques bring new challenges: robust quantitative measurements are needed to evaluate the findings from these studies, which can involve thousands of cells of varying response strength. Here we show that traditional measures of selectivity such as the orientation index (

In the visual cortex of all examined mammalian species (Hubel and Wiesel,

A number of measures have been devised to assess the strength and significance of orientation and direction selectivity for a given cell (Henry et al.,

New challenges—the advent of unbiased optical recording techniques such as 2-photon calcium imaging that sample all cells regardless of selectivity (Stosiek et al.,

Here we characterize the robustness of several measures of orientation and direction selectivity on simulated responses. We provide a recommendation for analysis methods for the principle questions that investigators usually ask: (1) How much orientation or direction selectivity does a cell exhibit? (2) Does a cell exhibit significant orientation or direction selectivity? (3) Has a manipulation introduced a significant change in the amount of orientation or direction selectivity at the population level? Further, we provide tables for statistical power, to estimate the amount of data that would be required to accurately answer these questions. These methods could in principle be extended to other sensory response properties or other modalities; however, their performance depends on the form of the underlying response function, so they may perform less reliably in domains aside from orientation and direction selectivity.

There is no standard coordinate system for indicating orientation or direction space. In this paper, we use “compass” coordinates, in which a horizontal bar moving upward is considered to be moving at 0°, and angles increase in a clockwise manner. Another common coordinate system is the Cartesian system, where 0° indicates a vertical bar moving to the right, and angles increase in a counterclockwise direction. One can transform between these two systems using the following equations:

Because there is no standard coordinate system for orientation and direction, and because some readers may be unfamiliar with orientation and direction, it is helpful to use pictures to indicate stimulus orientation and direction in slides and in published figures, as we do here.

In several equations, we express angles in terms of the sum of angles. For example, for a direction tuning curve we define the positive orthogonal orientation as follows: θ_{orth+} = θ_{pref} + 90°. Note that these angles are summed modulo 360° in direction space and modulo 180° in orientation space. For example, in direction space, 359° + 2° = 361° modulo 360° = 1°.

In this paper, we use _{pref_ori}_{orth}_{pref_ori}_{pref}_{null}_{pref.}_{pref_ori}_{orth}_{pref_ori}_{orth}_{pref}_{null}_{pref}_{null}_{orth}_{pref_ori}

The response at the preferred orientation _{pref_ori}_{pref}_{1}, θ_{2},…,θ_{n}, then we choose the response at the best θ_{i}. In other measures, we perform a fit to the tuning curve, and choose the maximum value of the fit as _{pref_ori}_{pref}

In Monte Carlo simulations of orientation and direction tuning curves, the “true” preferred angle and tuning widths were varied randomly so that a variety of tuning curve shapes were analyzed. Ranges were selected to correspond to typical values observed in V1 neurons. Each underlying “true” curve was a double Gaussian. The underlying angle preferences were chosen according to a uniform distribution between 0° and 360°. Tuning widths were chosen randomly according to a Gamma distribution with shape 3 and scale 6: σ = (Gamma(3,6)+10°)/1.18.

In many of the figures, we examined curves with 21 values of underlying _{p} = (_{n}_{p} = 10, _{n}

To calculate statistical power for simulated 2-condition experiments, we simulated underlying orientation or direction curves with exactly the

In each set of simulations, the simulated noise parameter is described. We used two types of noise. The most common type of noise, intended to capture the statistics of spikes recorded with an extracellular electrode, was a constant Gaussian noise value that was added to responses of all orientations on all trials. This constant value is often expressed as a percentage of the maximum response, which was usually 10 Hz. So, 20% noise means 2 Hz noise was added to individual trial measurements.

A more recent technique for recording neural responses is 2-photon imaging with Oregon Green BAPTA-1 AM (OGB-1AM) (Stosiek et al., ^{*} response magnitude.

Orientation selectivity has been observed in the visual cortex of every mammal that has been examined, including carnivores (Hubel and Wiesel,

Orientation selectivity is traditionally assessed by sweeping a bar or by drifting sinusoidal gratings across the cell's receptive field in different directions (Hubel and Wiesel,

_{pref}_{pref}), and a weaker response (_{null}_{null}). The cell responds less strongly to stimulation at either of the two orthogonal orientations (θ_{orth+} and θ_{orth+}). The cell's response decreases as the direction of the stimulus deviates from θ_{pref}; the difference between θ_{pref} and the angle that causes the response to drop to half (_{hh}_{hwhh}).

The responses of this example cell are shown on a graph in Figure _{pref}_{pref}), while the opposite direction is called the null direction (θ_{pref} +180°).

From the graphical tuning curve in Figure _{pref}_{null}_{orth+} and _{orth−}) at the orientations that are orthogonal (θ_{orth+} = θ_{pref} + 90°, θ_{orth−} = θ_{pref} − 90°) to the preferred orientation. This method has been employed in numerous studies, and we refer to it here as the

Note that it is not necessary to stimulate with directional stimuli in order to obtain a measure of orientation selectivity. Indeed, in many studies, the bars are drifted back and forth and the responses to each pair of opposite directions are averaged together. In this “orientation space,” the angle of stimulation ranges from 0° to 180°. We can calculate the orientation selectivity index in this case by using the preferred response (_{pref_ori}_{orth}_{orth} = θ_{pref} + 90°)

The _{orth}

In direction space, a

Another major notion of orientation or direction selectivity is the sensitivity of the response to the preferred angle. One can imagine measuring the amount one needs to change the orientation (or direction) angle from the preferred for the response to drop by some amount, such as by half (_{hh}_{hwhh} indicates how far in orientation space one must adjust the angle from the optimal to obtain half of the response height. This type of selectivity has been referred to as a cell's

Owing to the mathematical simplicity of the

However, the case of a model cell (

In single unit recording studies in adult animals, one often ignores cells with weak responses, but if one is conducting an imaging study of 100's of neurons, or developmental research in animals with weakly responsive cells, it is highly likely that some neurons will exhibit weak orientation selectivity. If the

We can improve the situation by plotting the responses to individual stimuli in a vector space. In Figure

_{ori}

The normalized length of this vector in orientation space is computed as follows:

where _{k}) is the response to angle θ_{k}. In direction space this length is the following:

The normalized vector length is related to a classic quantity in circular statistics called the

We use the abbreviation

This definition differs (by a factor of 2) from the classic definition of circular variance in direction space (Batschelet,

The vector lengths in orientation space (1-

When one records a neuron experimentally, one can only obtain a limited number of samples of the neuron's responses. One would like to use these sampled responses to make the best guess about the neuron's “true” properties, which cannot be examined directly but can only be inferred from experimental observations. Here we used Monte Carlo simulations to consider which index,

We created 21 model orientation tuning curves that ranged in “true” selectivity from 0 to 1 (Figure

The percentile distribution of empirical

While the results in Figures

We performed similar simulations for direction selectivity, comparing the empirical

The Monte Carlo simulation results presented in Figures

Experimentalists are also interested in knowing how many stimulus trials and stimulus angle steps should be presented to the animal in order to provide a quality estimate of the neuron's true orientation or direction selectivity. We performed Monte Carlo simulations where we systematically varied the single trial noise, number of stimulus trials, and the number of stimulus angles in order to understand how these factors influenced error in uncovering the “true”

When one suspects a cell is selective for stimulus orientation and/or direction, it is often important to verify this selectivity statistically. We need tools that allow us to answer the question “is a cell's selectivity for orientation/direction statistically significant?” In principle, one could simply measure a selectivity coefficient on each trial and perform statistics on this distribution of coefficients. However, the flaw in this analysis is in determining the null hypothesis with selectivity coefficients, and the flaw applies whether one uses

We have found that the best way to detect selectivity is to measure the magnitude of orientation or direction vectors (Figure

^{2}-test, which tests for whether the 2-dimensional mean of this distribution of orientation vectors is different from [0, 0].

For detecting orientation selectivity we use Hotelling's ^{2}-test, which is a multivariate generalization of Student's

For detecting direction selectivity, it is possible in principle to apply Hotelling's ^{2}-test to direction vectors. However, we have found that this method of testing for direction selectivity is quite insensitive because direction space is generally sampled too crudely to provide a reliable distribution of direction vectors. To address this problem, we developed a new test which we call the “direction dot product test.” This test uses both orientation vectors and direction vectors to assess the direction selectivity of a cell (Figure

In the direction dot product test, the first step is to obtain the orientation axis of the cell by calculating the angle of the average orientation vector. Next, we calculate the magnitude of the projection of each direction vector onto the orientation axis (this is what we call the “direction dot product” for each direction vector). This gives us a 1-dimensional distribution of direction dot product values, one value for each direction vector. Finally, Student's

The direction dot product reliably detects direction selectivity. Figure

Another objective that arises when one has a cell with selectivity for orientation/direction is to estimate the uncertainty of the measured selectivity parameters. Above we described tools for asking whether selectivity was significantly greater than zero. However, one might also like to obtain a measure of dispersion (e.g., standard deviation) or a confidence interval (e.g., standard error) for selectivity parameters. In principle, one could simply obtain this from the distribution of

One common question, especially in the era of 2-photon imaging where many cells are recorded simultaneously, is to ask whether one population of cells has different average selectivity than another population (or, equivalently, whether a population recorded at one point in time has different average selectivity than the same population recorded at another point in time). The approach is simple: Measure selectivity coefficients from each cell in the two populations, and perform a 2-sample

0.5 + 0.1 | 16/100 | 37/190 | 201/365 | 9/43 | 16/85 | 26/130 |

0.5 + 0.2 | 6/20 | 13/40 | 40/80 | 5/6 | 7/15 | 12/28 |

0.5 + 0.3 | 5/6 | 8/15 | 19/21 | 5/4 | 5/6 | 8/9 |

0.15 + 0.1 | 103/1k | 183/2k | 313/3k | 38/181 | 69/324 | 117/519 |

0.15 + 0.2 | 26/145 | 48/265 | 74/410 | 10/40 | 18/64 | 29/112 |

0.15 + 0.3 | 11/45 | 21/80 | 34/145 | 5/12 | 9/23 | 15/40 |

0.3 + 0.1 | 84/291 | 152/521 | 233/950 | 30/80 | 56/147 | 92/250 |

0.3 + 0.2 | 20/58 | 39/109 | 64/170 | 9/20 | 15/36 | 24/61 |

0.3 + 0.3 | 10/22 | 18/41 | 31/75 | 5/10 | 8/14 | 13/21 |

0.5 + 0.1 | 76/131 | 141/241 | 218/384 | 26/43 | 43/76 | 82/140 |

0.5 + 0.2 | 20/30 | 37/50 | 59/110 | 8/10 | 13/20 | 22/31 |

0.5 + 0.3 | 10/11 | 17/20 | 27/35 | 5/5 | 7/10 | 12/14 |

^{*} response. Angle step size was 22.5°. Note that many fewer cells are needed to evaluate changes in orientation and direction selectivity if 1-CirVar or 1-DirCirVar is used as a readout as compared to OI or DI

As an aside, one might wonder whether statistics on raw vectors could be used to answer this question. Since vector magnitudes correlate with selectivity, why not compare the vectors between the populations to see if selectivity has changed? The answer is that vector magnitudes, while they do correlate with selectivity, also correlate with tuning width and response magnitude (see Figure

Another common question is whether some specific response parameter differs between two cell populations. For example, one might wish to look for differences in preferred orientation between two populations. In this case, a vector-based test can be useful. Orientation vectors are affected by preferred orientation, so differences in preferred orientation lead to different distributions of vectors from the cells. Hotelling's ^{2}-test (specifically the 2-sample version of the test, analogous to the 2-sample Student's ^{2}-test in detecting differences in preferred orientation between two populations of cells.

However, this test must be used with caution. Vectors are affected by all response parameters including preferred orientation, tuning width, and response magnitude, so a positive result simply means that one or more of these parameters differs between the two populations; it cannot prove that the difference is in preferred orientation or any other single parameter. The test may be useful as a broad screen to detect generalized differences in response parameters. But if a difference in a specific response parameter is sought, the best method is to perform statistics with iterative fitting, as described below.

In order to address the question of how well a given population of neurons encodes the orientation or direction of a stimulus, it is often important to know the precise parameters of a cell's tuning function such as its tuning angle or tuning width. Previous work using Monte Carlo simulations (Swindale,

where _{pref} is the preferred orientation, _{p}_{ori}(x) = min(x, x − 180, x + 180), wraps angular difference values onto the interval 0° to 90°, and σ is a tuning width parameter. If we wish to only analyze the portion of the response above the offset, then the tuning width (half-width at half-height) is equal to

In direction space, we can use a double Gaussian with the following equation:

where _{pref} are defined as before, _{p}_{n}_{dir}(x) = min(x, x − 360, x + 360), wraps angular difference values onto the interval 0° to 180°, and σ is a tuning width parameter. Again, if we wish to only analyze the portion of the response above the offset, then the tuning width (half-width at half-height) is equal to

Although Gaussian fits are the best method for determining response parameters (Swindale,

_{p}

To prevent poor fitting, we use 2 _{p}_{n}_{pref} = θ_{M} where _{M}), _{p}_{n}

Using this ^{2}-test. The simulations of the model cells of varying direction selectivity in Figure

The relationships between fit quality and noise and number of stimulus trials and stimulus angles are plotted in Figure

Finally, one useful outcome of iterative fitting is that it can be used to estimate uncertainty in fit parameters and to do statistics on these parameters. The simplest method for doing this uses the Hessian matrix, which measures the steepness of the error function near the local minimum where the fit algorithm settles. The matrix is obtained by sampling the error function near the local minimum and measuring the partial second derivative of this function for each parameter; standard Matlab optimization tools produce the Hessian matrix as an output parameter. Once obtained, the Hessian matrix can be transformed to obtain standard errors of fit parameters, and these can then be used to perform statistics (Press et al.,

Unfortunately, the Hessian method does not work for fitting orientation and direction curves. Since the Hessian matrix represents the second partial derivatives of the error function, it can only be obtained when the error function is reasonably smooth. As described above, achieving adequate fits of orientation and direction data requires strict constraints on the fitting procedure. Because of these constraints, the error function is not smooth and thus a meaningful Hessian matrix generally cannot be obtained when fitting orientation and direction curves.

Another method for using iterative fitting to quantify uncertainty in parameters is the bootstrap method. In this method, samples of data are repeatedly selected at random, with replacement, and fits are performed to each sample. The distribution of parameters obtained in these fits provides a reasonable estimate of the parameter distribution in the underlying population (Press et al.,

In a previous study we employed the bootstrap method to estimate the distribution of preferred direction in individual cells from 2-photon recordings before and after extended exposure to a motion stimulus (Figure

One way we used this distribution was to detect significant direction selectivity. We quantified the “uncertainty” in direction preference, which is the percentage of simulations whose preferred direction differed from the mean preferred direction by more than 90°. This uncertainty can vary between 0 and 50%, so we interpret (

There are several drawbacks to the bootstrap method. First, the method is very computationally intensive, with a standard test requiring several days of computer time. More importantly, results obtained from the bootstrap method depend on a variety of factors that are not related to the data. Specifically, the outcome of the test depends on the fitting algorithm employed, the initial value used in the fit, and the constraints placed on the fit. Researchers using the bootstrap method must take care to record and publicize details about their technique so that others may reproduce their findings.

An alternative approach that generates confidence intervals is a Bayesian approach, such as that described in Cronin et al. (

Orientation and direction tuning are probably the most intensively studied response properties in the cortex. Historically, these studies have focused on cells with strong selectivity as determined by simple comparisons between preferred and non-preferred responses; cells without such obvious selectivity were often declared, simply, “unselective.” However, the advent of advanced techniques for recording and manipulating neurons requires us to investigate subtle differences between cells and to extend our analysis to cells with low selectivity. We need statistical tools that are suitable for addressing these subtle questions.

Traditional measures for quantifying orientation and direction selectivity rely on assigning the stimulus evoking the strongest response as the “preferred” stimulus for the cell and assign the opposite stimulus as “non-preferred.” The most commonly-used measures,

To obtain an accurate estimate of preferred and non-preferred stimuli, one must extrapolate between measured values. Vector-based methods effectively extrapolate measured responses by calculating the vector average of responses on each trial. Specifically, for quantifying selectivity, we recommend 1-

Vectors can also be used to assess whether a cell's selectivity is statistically significant. In this approach, we ask whether the 2-dimensional mean of orientation or direction vectors is significantly from zero. Specifically, Hotelling's ^{2}-test on orientation vectors is reliable for detecting orientation selectivity (Figure

In some cases, we need to probe beyond selectivity to ask about specific response parameters such as tuning angle or tuning width. Vectors can be used to detect differences in these parameters between two populations (Figure

Fitting also offers a tool for estimating the uncertainty of response parameters via the bootstrap method, where the data is randomly resampled multiple times with replacement and fits are performed to the resampled data. This method generates a distribution of values for each parameter which serves as an accurate estimate of the true distribution (Figure

Table

Quantifying the degree of orientation selectivity | 1- |

Quantifying the degree of direction selectivity | 1- |

Testing for significance of orientation selectivity | Hotelling's ^{2}-test on orientation vectors |

Testing for significance of direction selectivity | Direction dot product test on direction vectors |

Comparing orientation selectivity between two populations | 2-sample Student's |

Comparing direction selectivity between two populations | 2-sample Student's |

Screening for any difference in response parameters (e.g., preferred orientation, tuning width, peak height) between two populations | 2-sample Hotelling's ^{2}-test on orientation vectors |

Extracting response parameters such as tuning angle or tuning width | Fit data with Gaussian (for orientation data) or double Gaussian (for direction data) |

Quantifying the confidence/uncertainty of response parameters such as tuning angle or tuning width | Bootstrap method: Resample data with replacement, then fit resampled data with Gaussian (for orientation data) or double Gaussian (for direction data) |

Mark Mazurek, Marisa Kager, and Stephen D. Van Hooser performed analysis, Mark Mazurek and Stephen D. Van Hooser wrote the paper with comments from Marisa Kager.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.