^{1}

^{1}

^{2}

^{*}

^{1}

^{2}

Edited by: Robyn Grant, University of Sheffield, UK

Reviewed by: Quan Zou, University of Nevada, USA; Gerald Loeb, University of Southern California, USA

*Correspondence: Mitra J. Z. Hartmann, Mechanical Engineering Department, Biomedical Engineering Department, Northwestern University, 2145 Sheridan Road, Evanston, IL 60208, USA. e-mail:

This is an open-access article distributed under the terms of the

When an animal moves an array of sensors (e.g., the hand, the eye) through the environment, spatial and temporal gradients of sensory data are related by the velocity of the moving sensory array. In vision, the relationship between spatial and temporal brightness gradients is quantified in the “optical flow” equation. In the present work, we suggest an analog to optical flow for the rodent vibrissal (whisker) array, in which the perceptual intensity that “flows” over the array is bending moment. Changes in bending moment are directly related to radial object distance, defined as the distance between the base of a whisker and the point of contact with the object. Using both simulations and a 1×5 array (row) of artificial whiskers, we demonstrate that local object curvature can be estimated based on differences in radial distance across the array. We then develop two algorithms, both based on tactile flow, to predict the future contact points that will be obtained as the whisker array translates along the object. The translation of the robotic whisker array represents the rat's head velocity. The first algorithm uses a calculation of the local object slope, while the second uses a calculation of the local object curvature. Both algorithms successfully predict future contact points for simple surfaces. The algorithm based on curvature was found to more accurately predict future contact points as surfaces became more irregular. We quantify the inter-related effects of whisker spacing and the object's spatial frequencies, and examine the issues that arise in the presence of real-world noise, friction, and slip.

As an animal moves through the environment, the spatial and temporal gradients of sensory data it acquires are related through the velocity of its moving sensory surfaces. This relationship is represented by the “complete derivative” (Munson et al.,

In the field of visual neuroscience, the complete derivative has been termed the “optical flow” equation. The optical flow equation relates spatial and temporal intensity (brightness) gradients to the velocity of the animal (Barron et al.,

Notably, the optic and tactile flow equations are typically used by assuming that the animal makes use of spatial and temporal gradients of brightness (vision) or strain energy (tactile) to compute the velocity of its sensor array.

We recently proposed a complementary scheme: namely, if the animal already knows its own velocity, then it can use the complete derivative to predict future sensory data (Gopal and Hartmann,

In the present work, we used the rat vibrissal system as a model to examine the plausibility of using the complete derivative to predict upcoming sensory data. Rats actively brush and tap their whiskers (vibrissae) against objects to tactually explore the environment. During detailed exploration of objects, rats move their vibrissae rhythmically, between 5 and 25 Hz (Welker,

The present work was designed to investigate tactile flow across the vibrissal array in this type of translational navigation behavior. We used both simulations and a 1 by 5 array of artificial vibrissae (hardware) to investigate the plausibility of using the complete derivative to predict upcoming whisker-object contact points.

Our initial investigations were performed in simulation, and we then investigated the real-world issues that arise with implementation on a 1 × 5 array of robotic whiskers.

Radial object distance is defined as the Euclidean distance between the base of a vibrissa and the point at which it makes contact with an object. Previous work has shown that an object's contour can be extracted by continuous rotation of a vibrissa against an object (Kaneko et al.,

During wall-following behavior, however, rats do not always whisk, but sometimes keep their vibrissae protracted against the wall. This behavior is better modeled as a relative translation between vibrissa and object, rather than as a rotation. Although Solomon and Hartmann (

In the present study, we experimentally validate the translation equations developed in Solomon and Hartmann (

As shown in previous work (Kaneko et al., _{0} to the initial contact point on an object can be calculated using
_{0} is a small pushing angle beyond initial contact (typically about 3°), and _{0} is the bending moment at the vibrissa base.

Once the radial distance _{0} to the initial contact point is calculated, the whisker undergoes either a small rotation (_{i}) can then be calculated based on estimates of radial distance at previous time steps.

The procedures for calculating _{i} after a translation or rotation are quite similar, but they differ in the calculation of the magnitude of the vector _{0}, Solomon and Hartmann (_{i−1} is the radial distance at the previous time step (before rotation), _{i−1} is the angle between the vibrissa base at time _{0} and the base at time _{i−1} (after rotation).

In contrast,

The value of _{i}, regardless of whether the whisker was translated or rotated. For a detailed explanation of this calculation, please refer to Figure

The local slope (μ) and the local curvature (κ) of an object can be calculated based on radial distance measurements. To calculate either μ or κ, we first define our coordinate system as shown in Figure _{i} is the distance the base-point of the whisker has translated in timestep _{i} at each time step:

_{rat}, and the arc _{rat}. The distance _{i} is the distance in the direction of _{i} and _{i-1}. _{i-1} is the distance in the direction of _{i-1} and _{i-2}. _{i} is the radial distance measure at the ^{th} time step, _{i} is an angle measure of the change in direction of _{rat}, and μ_{i} represents the local slope of the object.

Slope μ_{i} is defined by the differences between radial distance measures (_{i} and _{i−1}) divided by the distance traveled (_{i}). Each

Calculation of curvature is similar, but requires us to first calculate the slope μ_{i−1} obtained in timestep

Curvature is then defined using both μ_{i} and μ_{i−1}:

This equation is simply the discretized version of the definition of curvature measured in 2D Cartesian coordinates. Notably, the commonly-used simplification of Equation 6 that assumes μ_{i} is small compared to unity (and therefore reduces the equation to just the numerator) was empirically found to be inaccurate. The small slope assumption does not always hold in our calculations.

When the rat's heading does not change significantly between timesteps, _{i} and _{i−1} are both zero and Equation 4 can be simplified to:
_{i} and _{i−1}) over the distance traveled (_{i}). Equation 5 can be similarly simplified and Equation 6 remains the same.

So far, we have described the sensory data obtained by a single whisker over time. Equations 4–7 apply equally well, however, to multiple whiskers making simultaneous contact with an object, at different locations on the object. In this case, the subscript ^{th} whisker, instead of the ^{th} timestep. Figure

Equations 1–7 define the calculations used to find radial distance, local object slope, and local object curvature. Using these calculated values, we now show that it is possible to make predictions about the sensory data the animal will receive.

Because animals control the movements of their limbs, they control the velocity with which sensory data flow over their sensory surfaces (Gopal and Hartmann,

In this equation,

In this paper, we choose to represent “tactile flow” through radial distance of contact along the rat whisker because radial distance is directly related to changes in bending moment at the vibrissal base (Kaneko et al., _{i+1} is the radial distance of the future point of contact, _{i} is the current radial distance, _{i+1} and _{i} are the times at which the radial distances are measured, _{i} is the distance traveled in the last time step, and μ_{i} is the current slope. Rearranging the equation and substituting μ_{i} as in the last term above yields:

Thus it is clear that future radial distance can be estimated from the current slope.

If the slope of the object is changing, we expect the current curvature of the object to be able to give a better estimate of future radial distance. To use curvature, we first predict the future slope of the object, and then use that predicted slope to predict future radial distance. The future slope μ_{i+1} is found by choosing μ as the parameter in Equation 8, as shown in Equation 11:

The simplification that _{i} as shown in Equation 6, yielding:
_{i+1} is the local object slope at the future point of contact, μ_{i} is the current local object slope, _{i} is the current local object curvature, and _{i+1} and _{i} are the times at which the radial distances used to calculate slope and curvature are measured. When Equation 12 is substituted into Equation 10 (with μ_{i +1} replacing μ_{i}), we obtain an equation that estimates future radial distance based on both current slope and current curvature:

Finally, in the type of wall following behavior and environmental exploration in which prediction would be most useful to a rat, it is more likely that significant changes in calculated local object slope and local object curvature are due to measurement error than due to abrupt changes in the object. To decrease the sensitivity of predicted radial distance to measurement error, we average the local slope and curvature as follows:
_{avg} is the average slope, κ_{avg} is the average curvature, _{i} is the local object slope between each radial distance measurement, and κ_{i} is the local object curvature between each calculated slope value. Substituting Equations 14 and 15 into Equations 10 and 13 yields:

As illustrated in Figure

Radial object distance is defined as the Euclidean distance between the base of a vibrissa and the point of object contact (Szwed et al.,

The present work used a single horizontal row of a five by five array of vibrissae (Figure

A linear actuation system was built to model the forward (translational) movement of a rat as it explores its environment. Because the vibrissa array was tethered with power and signal cables, it was easier to translate an object past the stationary vibrissa array than it was to translate the array past the object. In the present work, these two paradigms are equivalent because we are concerned only with relative velocity between the array and object. This study did not explore methods to distinguish between self-generated versus external movements.

As shown in Figure

The two strain gauges for each sensing dimension on each vibrissa were arranged in a half Wheatstone bridge so the bending of the vibrissa created a change in voltage at the output of the circuit. The resting output was zeroed with a potentiometer. The actual value of the voltage output depended on circuit parameters such as gain, so it was necessary to calibrate the voltage output to the curvature at the base of the vibrissa. To calibrate the voltage to the curvature, the vibrissa was rotated against a peg placed at a known distance from the vibrissa base. For cylindrical homogenous vibrissae, curvature and moment differ only by a scaling factor. Given the known angular deflection of the cylindrical vibrissa and the distance between the base and peg, Solomon and Hartmann have shown that the moment can be calculated based on Euler-Bernoulli beam theory (Solomon and Hartmann,

Once a vibrissa has been calibrated, the voltage recorded from that vibrissa is converted to moment at the base of the vibrissa. The radial distance _{0} to the initial contact point is calculated using Equation 1.

To quantify the accuracy of radial distance prediction using each method, prediction error was defined as:
_{pred} is the prediction error, _{i} is the predicted radial distance for each sample, and _{i} is the measured radial distance for each sample. _{pred}, which we define as prediction error, is the mean absolute error of the prediction. It is mean absolute percent error (MAPE) when multiplied by 100. For simulated results, _{i} is chosen to be accurate to machine precision. This choice results in a prediction error that solely measures the accuracy of the prediction algorithm for a given set of parameters. In hardware, _{i} is calculated using the measured bending moment _{pred} is affected both by measurement error as well as the accuracy of the prediction algorithm. In practice, however, errors in radial distance extraction (measurement errors) were small compared to errors in prediction.

As described in

In all simulations in this section, we assume “perfect” radial distance extraction at time 1, and then calculate slope and curvature to predict radial distance at time 2. In other words, we do not simulate whisker deflection. The goal of this section is to verify that the use of either slope or curvature is sufficient for the prediction of future radial distance. Simulation will also show whether the increased mathematical complexity required by the use of local curvature leads to significantly increased predictive accuracy over the simpler equation for local slope.

In these simulations, the data collection method (i.e., single vibrissa or multiple vibrissa) is irrelevant, as the main difference between the two methods is spatiotemporal scale—the simulated objects can be made to arbitrary size and the sampling rate can be increased or decreased arbitrarily.

To quantify prediction error from the use of Equations 16 and 17, we simulated translation of the whiskers past several differently sized cylinders. We then quantified how well the predicted values matched actual values. The results of this simulation are shown in Figure

Figure

In Figure

The second test object (Figures

In summary, these simulation results demonstrate that local object curvature more accurately predicts future contact points on regions of objects with no distinct edges, but that when objects have distinct edges, prediction using local object slope is more accurate. Of course, this in turn raises the question of what it means for an object to have a “distinct edge.” Mathematically, it must be an edge in the sense that there is a discontinuity in the curvature between measurements, which is clearly related to the spatial scale of the object relative to the spacing of the whiskers. We investigate this in the next section.

The spacing between the whiskers on the object surface places limits on the maximum curvature that can be sensed. Equation 19 defines κ_{max}, the maximum discriminable curvature:
_{max},

In the present work, we assumed that the spacing between the whiskers on the object surface was approximately equal to the spacing between them at their base. This is a reasonable approximation for the present work, in which the whiskers are parallel to each other, but is unlikely to be valid for the real rat.

Figure

To demonstrate the effect of relative vibrissa spacing on prediction error, prediction was simulated over hyperbolic spirals of various sizes. The hyperbolic spiral was chosen as the test object because it can be translated past the vibrissa array in such a way that curvature decreases approximately linearly. In these simulations, the multiple-vibrissa prediction method was used, with vibrissa spacing set to 1 cm. The size of the spiral test object was decreased for each case to illustrate the effect of relative vibrissa spacing.

Figure

We next aimed to validate these simulation results in hardware. However, before we could do so, we needed to experimentally validate the translational sweep algorithm proposed by Solomon and Hartmann (

Having validated the translational sweep algorithm, we next aimed to test the two prediction methods in hardware. Figure

Neurons in the trigeminal nuclei have receptive fields that include multiple vibrissae, which may enable the rat to estimate the surface gradient at a single time point. For this technique to be viable, the chosen vibrissae must all be touching the object at once. The spacing between adjacent vibrissae in the final design of the array (Figure

The results of the previous two sections show that prediction using local object curvature can accurately predict future contact points for an object with constant curvature. In the world, however, such objects are rarely found. We did not implement prediction using the multiple vibrissa method on this object because the main difference between the spatial scales over which we were making abrupt changes was better represented by the single vibrissa method.

To show the effect of abrupt changes in object curvature on prediction, a simple case was examined. Figure

It can be seen from Figure

We tested the hardware vibrissa array with a section of a hyperbolic spiral that had an approximately linear decrease in curvature as the object was translated past the whiskers.

Figure

This paper has demonstrated that simple algorithms can be used to predict future contact points on an object. Prediction is accurate for objects that have constant and/or gradually changing curvature as long as the distance between vibrissae is small relative to the change in curvature. Abrupt changes in object curvature result in jumps in absolute percent error of the prediction.

Two different algorithms for prediction were described in the results section. Prediction using local object slope is only accurate when the curvature being measured does not change much between estimated contact points, but it can accommodate for abrupt changes in curvature (e.g., an edge). Prediction using local object curvature was shown to be more accurate for both objects with constant curvature and objects with gradual changes in curvature.

In all cases, vibrissa spacing makes a difference in accuracy of prediction. This result is hardly surprising, since the limiting factor is essentially sensor resolution. The main advantage of widely spaced vibrissae is the ability to predict further ahead in space. Since information about the sensed object is spread over a wider distance, it is more likely that the estimated curvature will be a reflection of the overall curvature of the object rather than a measurement of a local deviation from the actual object curvature.

These observations lead to the hypothesis that vibrissa spacing represents a trade-off between accuracy and predictive utility. For wall following, we anticipate that the rat will protract its vibrissae far forward and maximize spacing between the tips, because predictive sensing over a large spatial scale is important. For edge detection tasks, accuracy becomes more important so we anticipate that the vibrissal tips will be spaced more closely together.

Motion detection, as well as the ability to distinguish self-movement from environmental movement, is critical to animal survival. Behaviors such as escape or predation must link motion detection to immediate motor action. For these behaviors, the quality of the sensory data obtained is largely irrelevant, as long as it is sufficient to trigger the appropriate motor action. We suggest that a mismatch between predicted and actual sensory input may serve to direct attention.

In this work, we have presented specific examples of how calculating terms of the total derivative might be used by a moving rat to track an object within its vibrissal sensory array. If an object is moving in the vibrissal field, there will be a mismatch between actual and predicted input that is exactly equal to how the world is changing in time. In the next time step, the animal can use this mismatch to compute the relative velocity between its own movements and movements in the world. In other words, the animal can compare predicted and actual sensory data obtained to estimate the quantity

During wall-following behavior, a rat maintains a small separation between itself and the wall while traveling at a relatively high velocity. The rat can maintain this separation even when walls curve, however, since the rat has mass, and it moves at a high velocity, changing direction takes time and energy. Following a curving wall would be easier if the rat could predict the future wall profile using its vibrissae. More specifically, if a rat could use just radial distance measures to predict the upcoming wall contour, it could start to change direction sooner, reducing inertial delays and saving energy. Encoding by its vibrissa array to determine where the wall was likely to be in the future, it would save energy and allow the rat to travel along the wall at a higher velocity. With these prediction algorithms that prediction could be accomplished at a very low level of processing. The algorithms presented here suggest that this type of prediction could be achieved with a very low level of computation.

An examination of the results presented in Figure

The artificial whiskers used in this work were cylindrical, but real rat whiskers taper linearly. Tapering the whisker confers at least two advantages. First, whisker taper increases sensitivity to small contact forces (Williams and Kramer, _{push}, the angle through which the whisker has rotated against the object (Solomon and Hartmann,

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

This work was funded by NSF award IOS-0818414 and NSF CAREER award IOS-0846088 to Mitra J. Z. Hartmann and NSF EFRI-0938007 (Liu, PI). Christopher L. Schroeder was sponsored by NSF IGERT # DGE-0903637: Integrative Research in Motor Control and Movement (Hale, PI).