^{1}

^{2}

^{1}

^{3}

^{1}

^{3}

^{4}

^{1}

^{3}

^{4}

^{*}

^{1}

^{2}

^{3}

^{4}

Edited by: Thomas Nowotny, University of Sussex, United Kingdom

Reviewed by: Changsong Zhou, Hong Kong Baptist University, Hong Kong; Jorge F. Mejias, University of Amsterdam, Netherlands

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

Reliable propagation of slow-modulations of the firing rate across multiple layers of a feedforward network (FFN) has proven difficult to capture in spiking neural models. In this paper, we explore necessary conditions for reliable and stable propagation of time-varying asynchronous spikes whose instantaneous rate of changes—in fairly short time windows [20–100] msec—represents information of slow fluctuations of the stimulus. Specifically, we study the effect of network size, level of background synaptic noise, and the variability of synaptic delays in an FFN with all-to-all connectivity. We show that network size and the level of background synaptic noise, together with the strength of synapses, are substantial factors enabling the propagation of asynchronous spikes in deep layers of an FFN. In contrast, the variability of synaptic delays has a minor effect on signal propagation.

Information in the brain is encoded by either the number of spikes in a relatively long time window, i.e., rate code, or by their precise timing, i.e., temporal code (Abeles et al.,

The reliable propagation of synchronous spikes (temporal code) is well-understood and relatively easy to implement in computer models (Kumar et al.,

Several studies have addressed the conditions required for spike-rate propagation (Shadlen and Newsome,

Neuronal networks with feedforward connections consisting of excitatory neurons are thought of as the model for linking upstream neurons with downstream neurons, either across different layers within the same cortical region or between different cortical regions. Recent studies have capitalized on the role of recurrent connections in reliable transmission of synchronous and asynchronous spikes (Joglekar et al.,

Despite undoubtedly significant impacts of biologically realistic network architectures in conveying information across/between layers, neuronal networks with feedforward connections play a substantial role in understanding the mechanisms of faithful propagation of different types of spikes (Kumar et al.,

In this paper, we focus to systematically explore the necessary conditions underlying which information of time-varying asynchronous spikes can be reliably transmitted in deep layers of an FFN. Using optimal synaptic weights (Faraz et al.,

Network size, level of background synaptic noise, and variability of synaptic delays are the main factors that were investigated in a feedforward architecture. We created an FFN composed of excitatory neurons, modeled by leaky integrate and fire (LIF) model, receiving shared input from the previous layer plus background synaptic noise. We calculated coding fraction—representing goodness of propagation of time-varying asynchronous spikes—for different levels of background synaptic noise, network size, and variability of synaptic delays.

To test the effect of each factor, we estimated synaptic weights using a reduced network model whose function, in the propagation of a shared input, is equivalent to an FFN with all-to-all connectivity [see (Lankarany,

The slow signal was delivered to the neurons in the first layer of an FFN and modeled by an Ornstein-Uhlenbeck (OU) process of the time constant of 50 msec. This slow signal can, for example, represent the luminance of a natural stimulus (Lankarany et al.,

where ξ is a random number drawn from a Gaussian distribution with 0 average and unit variance. τ is the time constant, μ and α indicate the mean and standard deviation of variable

Background synaptic noise was modeled by an OU process of the time constant of 5 msec (Destexhe et al.,

We used leaky integrate and fire (LIF) model. The dynamics of membrane potential is expressed as follows.

where _{L} = −70 mV, _{V} = 10 msec. _{inj} indicates the injected current (slow signal as the stimulus of the first layer). Spike occurs when V≥Vth, where _{th} = −40 mV and the reset voltage is −90 mV. For the neurons in the subsequent layers, _{inj} is equal to the total presynaptic input (weighted by synaptic strengths) plus independent synaptic noise. A double exponential function of τ_{rise} = 0.5 _{fall} = 5

Our computational study was performed by varying the number of neurons in an FFN. Network sizes were varied in a range of [50, 100, 200, 300, 400, 500, 750, 1000].

Variability in synaptic delays introduced heterogeneities in the network. This variability can be interpreted as an un-equal distance between pre- and post-synaptic neurons (see Discussion). We modeled such variability by a Gaussian distribution whose mean represents the synaptic delay (3 msec) and standard deviation indicates the level of variability across neurons. The std of synaptic delays varies within an interval of (0:0.25:1.5) msec, which is in agreement with small variations of synaptic latencies in the cortex (Boudkkazi et al.,

The instantaneous firing rate was calculated by convolving superimposed spikes (of each layer of an FFN) with a Gaussian kernel. To achieve a consistent comparison between the firing rates across layers, we used a Gaussian kernel of width = 25 msec. This kernel width was near to optimal for spikes in the first layer of the FFN (regardless the network size and noise level). We used a method proposed in Shimazaki and Shinomoto (

To quantify how much network size and the level of background synaptic noise effect the propagation of asynchronous spikes in a network, we used coding fraction (CF) that represents how good the instantaneous firing rate of the second layer tracks that of the first layer. CF is calculated as follows.

where _{2} indicates the norm 2. CF lies within [−1, 1], where 1 represents perfect transmission.

Besides, in order to validate whether CF is equivalent to an information-theoretic measure, we calculated Kullback–Leibler (KL) divergence measures between the instantaneous firing rate of the second and the first layers of the FFN. The Kullback–Leibler (KL) divergence (Timme and Lapish, _{1}) and the second (_{2}) layers. The KL divergence is defined as (Perez-Cruz,

where _{Fr1} and _{Fr2} are the probability distributions of the firing rates, namely, _{1} and _{2}, respectively. To calculate these probability distributions, we utilized a non-parametric estimation method to approximate them using normal kernel smoothing (Bowman and Azzalini,

where _{KL} is non-negative (≥ 0) and non-symmetric in _{2}) and _{1}). _{KL} is equal to zero if _{2}) and _{1}) match exactly, and can potentially equal infinity if there is no similarity between the two distributions. In other words, the KL measure is equal to zero if the firing rates of both layers have the same statistical characteristics, implying a perfect information propagation (across two layers).

We calculated CF and KL divergence for different levels of background synaptic noise (for

Similar to Lankarany et al. (

To explore the effects of network size and the level of background synaptic noise in the propagation of slowly time-varying asynchronous spikes, we vary these parameters in an FFN consisting of two layers and calculate the coding fraction (CF). In addition, the number of synchronous events (see Methods) is quantified to provide more numerical characterizations of the roles of the above parameters in signal propagation.

A recent study (Barral et al.,

CF increases for medium and large network sizes with moderate to high levels of background noise. However, CF is relatively low outside these ranges; it decreases for small network sizes regardless of the noise level. More precise characterization of CF against different values of network size and noise level (see ^{pA}.

The number of synchronous events increases for medium- and large-size FFNs with low levels of noise (also note that the signal is not well represented in small-size FFNs with low noise). Thus, one can predict that more synchronous events would be generated in the subsequent layers for low levels of background synaptic noise.

To visually inspect the effects of network size and noise std, the instantaneous firing rates of the first and the second layers are shown in

To better understand the effects of network size and noise level on the transmission of asynchronous spikes within the first two layers of an FFN, CF is plotted in

1-D CF. CF vs. different network sizes when noise std is equal to 30 pA (

Simulating an FFN with the optimal network size and level of synaptic noise enhances the propagation of asynchronous spikes in deeper layers. To evaluate the performance of an FFN with the optimal parameters in the propagation of asynchronous spikes in deeper layers, we show the raster plot and the instantaneous firing rate of an FFN up to five layers in

Reliable propagation of slowly-time varying asynchronous spikes in 5 layers [raster plot & instantaneous firing rate (Gaussian kernel of width 25 msec)] of FFNs with optimal values of network size (500), level of synaptic noise (30 pA), and synaptic strength. Note: the delay between layers is compensated in these curves.

In addition to synaptic noise that provides heterogeneities in an FFN [or a locally connected random network (Mehring et al.,

To test whether the estimated weights are biologically realistic, we validate if the corresponding postsynaptic potential (PSP) lies within a biologically-reasonable range. We show in

Heat map of postsynaptic potentials (corresponding to synaptic weights) for different network sizes and levels of synaptic noise.

Improper selection of synaptic weights for an FFN results in unstable propagation of asynchronous spikes. Systematically varying synaptic strengths from moderately weak to moderately strong synapses, we find a transition between attenuation mode, where transmission of time-varying rates failed, to an amplification mode where the average firing rate increased at the subsequent layers (

Improper synaptic weights result in either attenuation or amplification of the slow signal in the subsequent layers. Instantaneous firing rates of an FFN with

It is to be noted that the time constants of excitatory synapses and neurons' membrane potential influence the estimated synaptic weights. Chan et al. (

In biologically-realistic scenarios, presynaptic inputs are delivered from neurons with different distances to the target (postsynaptic) neuron. In the context of signal propagation, for example, Joglekar et al. (

CF vs. std of synaptic delays in a two-layer FFN with

We construct logistic maps of the firing rates in the first and the 5th layers of an FFN (

Logistic map showing output rate (firing rate in layer 5) vs. input rate (firing rate in layer 1). The red dashed line has a slope of 1.

Necessary conditions for reliable propagation of time-varying asynchronous spikes in FFNs were investigated in this paper. Previous studies have addressed those conditions for transmission of synchronous spikes as well as the mean of firing rate of asynchronous spikes. However, these conditions barely remain valid for transmission of time-varying asynchronous spikes whose instantaneous rate of changes represents information of slow fluctuations of the stimulus. In this paper, we investigated the necessary conditions for reliable transmission of asynchronous spikes in an FFN. Specifically, we explored the effect of network size, level of background synaptic noise, and the variability of synaptic delays in an FFN with all-to-all connectivity. We demonstrated that network size and the level of background synaptic noise, together with optimal synaptic weights, were substantial factors that cooperatively enable the propagation of asynchronous spikes in deep layers of an FFN. Nevertheless, the variability of synaptic delays had a minor effect on signal propagation. Varying these factors systematically, we found that an FFN of network size of {400–500} with the level of synaptic noise in a range of {20–30} pA and proper synaptic weights transmits time-varying asynchronous spikes reliably.

Reliable propagation of a rate code is challenging: even weak pairwise correlations in the spike timing can notably deteriorate the fidelity of the rate code (Kumar et al.,

Despite the focus of this paper on the use of networks with feed-forward connections, recent studies demonstrated the feasibility of reliable signal propagation in recurrent networks (Joglekar et al.,

Thus, studying conditions of reliable signal propagation in an FFN helps better understanding the underlying mechanisms of information propagation in more biologically realistic scenarios in which the dynamics of an FFN interact with that of the embedding recurrent network.

We showed that the impact of heterogeneous synaptic delays on the propagation of asynchronous spikes in an FFN was not significant. However, the heterogeneity in synaptic delays can alter the dynamics of a recurrent neural network and have a substantial effect on signal transmission. Here, we discuss three scenarios that capitalize on the substantial but ambiguous impact of the variability of synaptic delays in signal propagation. First, the heterogeneity in synaptic delays might have contrasting effects if applied to excitatory and inhibitory neurons. In a strongly recurrent network with excitatory synapses, this heterogeneity can reduce the likelihood of synchronization. In contrast, it can alter the tight balance of excitatory and inhibitory inputs—necessary for a reliable signal propagation—in a recurrent network with lateral inhibition (Joglekar et al.,

Stochastic resonance in the neural systems was first observed in the response of the neuronal network to a weak periodic signal (Gluckman et al.,

Similar to noise-induced resonance, an optimal number of elements in a biological system can maximize the regularity in the emitted signal (in the presence of optimal level of noise), i.e., system-size coherence resonance (Toral et al.,

The datasets generated for this study are available on request to the corresponding author. Codes are available at

NH: Conceptualization and analysis, programming, and writing the manuscript. MR and SF: Analysis and programming and contributing in the writing of the revised manuscript. MP: Supervising the study and revising the manuscript. ML: Conceptualization and analysis, design the study, programming, and writing the manuscript.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The authors declare that some preliminary results of this paper have been released as a pre-print (Lankarany,

The Supplementary Material for this article can be found online at: