^{1}

^{†}

^{1}

^{†}

^{2}

^{2}

^{3}

^{1}

^{4}

^{‡}

^{5}

^{‡}

^{2}

^{*}

^{‡}

^{1}

^{3}

^{*}

^{‡}

^{1}

^{2}

^{3}

^{4}

^{5}

Edited by: Paul Miller, Brandeis University, United States

Reviewed by: Cees van Leeuwen, KU Leuven, Belgium; Andreas Knoblauch, Hochschule Albstadt-Sigmaringen, Germany

*Correspondence: Henry Markram

Kathryn Hess

†These authors have contributed equally to this work.

‡Co-senior author.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

The lack of a formal link between neural network structure and its emergent function has hampered our understanding of how the brain processes information. We have now come closer to describing such a link by taking the direction of synaptic transmission into account, constructing graphs of a network that reflect the direction of information flow, and analyzing these directed graphs using algebraic topology. Applying this approach to a local network of neurons in the neocortex revealed a remarkably intricate and previously unseen topology of synaptic connectivity. The synaptic network contains an abundance of cliques of neurons bound into cavities that guide the emergence of correlated activity. In response to stimuli, correlated activity binds synaptically connected neurons into functional cliques and cavities that evolve in a stereotypical sequence toward peak complexity. We propose that the brain processes stimuli by forming increasingly complex functional cliques and cavities.

How the structure of a network determines its function is not well understood. For neural networks specifically, we lack a unifying mathematical framework to unambiguously describe the emergent behavior of the network in terms of its underlying structure (Bassett and Sporns,

Algebraic topology (Munkres,

We developed a mathematical framework to analyze both the structural and the functional topology of the network, integrating local and global descriptions, enabling us to establish a clear relationship between them. We represent a network as a directed graph, with neurons as the vertices and the synaptic connections directed from pre- to postsynaptic neurons as the edges, which can be analyzed using elementary tools from algebraic topology (Munkres,

Networks are often analyzed in terms of groups of nodes that are all-to-all connected, known as

We applied this framework to digital reconstructions of rat neocortical microcircuitry that closely resemble the biological tissue in terms of the numbers, types, and densities of neurons and their synaptic connectivity (a “microconnectome” model for a cortical microcircuit, Figures

We found a remarkably high number and variety of high-dimensional directed cliques and cavities, which had not been seen before in neural networks, either biological or artificial, and in far greater numbers than those found in various null models of directed networks. Topological metrics reflecting the number of directed cliques and cavities not only distinguished the reconstructions from all null models, they also revealed subtle differences between reconstructions based on biological datasets from different animals, suggesting that individual variations in biological detail of neocortical microcircuits are reflected in the repertoire of directed cliques and cavities. When we simulated microcircuit activity in response to sensory stimuli, we observed that pairwise correlations in neuronal activity increased with the number and dimension of the directed cliques to which a pair of neurons belongs, indicating that the hierarchical structure of the network shapes a hierarchy of correlated activity. In fact, we found a hierarchy of correlated activity between neurons even within a single directed clique. During activity, many more high-dimensional directed cliques formed than would be expected from the number of active connections, further suggesting that correlated activity tends to bind neurons into high-dimensional active cliques.

Following a spatio-temporal stimulus to the network, we found that during correlated activity, active cliques form increasingly high-dimensional cavities (i.e., cavities formed by increasingly larger cliques). Moreover, we discovered that while different spatio-temporal stimuli applied to the same circuit and the same stimulus applied to different circuits produced different activity patterns, they all exhibited the same general evolution, where functional relationships among increasingly higher-dimensional cliques form and then disintegrate.

Networks of neurons connected by electrical synapses (gap junctions) can be represented as undirected graphs, where information can flow in both directions. Networks with chemical synapses, which impose a single direction of synaptic communication from the pre- to the postsynaptic neuron (Figures

When the direction of connections is not taken into account, a great deal of information is lost. For example, in the undirected case, there is only one possible configuration for a clique of four fully connected neurons (Figure ^{6} = 729 possible configurations, as each of the six connections can be in one of three states (i → j, j ← i, or i ↔ j connection types; Figure

A clique with reciprocal connections contains two or more cliques consisting only of uni-directional connections (Figure ^{6} possible configurations of four fully connected neurons, which are of two types: those that contain cycles (40 configurations; Figure

We analyzed 42 variants of the reconstructed microconnectome, grouped into six sets, each comprised of seven statistically varying instantiations (Markram et al.,

To compare these results with null models, we examined how the numbers of directed simplices in these reconstructions differed from those of artificial circuits and from circuits in which some of the biological rules of connectivity were omitted (see Section 4.4). For one control, we generated five Erdős-Rényi random graphs (ER) of equal size (~31,000 vertices) and the same average connection probability as the Bio-M circuit (~0.8%; ~8 million edges) (Figure

Simplices of high dimensions (such as those depicted in Figure

To test whether the presence of large numbers of high-dimensional directed simplices is a general phenomenon of neural networks rather than a specific phenomenon found in this part of the brain of this particular animal and at this particular age, we computed the numbers of directed simplices in the

To understand the simplicial architecture of the microcircuit, we began by analyzing the sub-graphs formed only by excitatory neurons, only by inhibitory neurons, and only in individual layers by both excitatory and inhibitory neurons. Restricting to only excitatory neurons barely reduces the number of simplices in each dimension (Figure

The large number of simplices relative to the number of neurons in the microcircuit implies that each neuron belongs to many directed simplices. Indeed, when we counted the number of simplices to which each neuron belongs across dimensions, we observed a long-tailed distribution such that a neuron belongs on average to thousands of simplices (Figure

The presence of vast numbers of directed cliques across a range of dimensions in the neocortex, far more than in null models, demonstrates that connectivity between these neurons is highly organized into fundamental building blocks of increasing complexity. Since the structural topology of the neural network takes into account the direction of information flow, we hypothesized that emergent electrical activity of the microcircuitry mirrors its hierarchical structural organization. To test this hypothesis, we simulated the electrical activity of the microcircuit under

Stimuli, configured as nine different spatio-temporal input patterns (Figure

To avoid redundant sampling when testing the relationship between simplex dimension and activity, we restricted our analysis to

The neurons forming maximal 1-simplices displayed a significantly lower spiking correlation than the mean (Figure

To determine the full extent to which the topological structure could organize activity of neurons, we examined spike correlations between pairs of neurons within individual simplices. These correlations increased with simplex dimension (Figure

We found that correlations were significantly higher for the last two neurons in the simplex, suggesting that shared input generates more of the pairwise correlation in spiking than indirect connections in directed simplices (^{−6}, all dimensions except 1D). Moreover, the spiking correlation of the source and sink neurons was similar to the correlation of the first and second neurons (Figure

Simplices are the mathematical building blocks of the microcircuitry. To gain insight into how its global structure shapes activity, it is necessary to consider how simplices are bound together. This can be achieved by analyzing the

_{5}) for seven instances of reconstructed microcircuits based on five different biological datasets (Bio 1-5).

To analyze directed flag complexes we computed two descriptors, the _{n}, indicates the number of n-dimensional cavities. For example, in Figure _{2} = 1) enclosed by the eight triangles; if an edge were added between any two non-connected nodes, then the geometric object realizing the corresponding flag complex would be filled in with solid tetrahedra, and the cavity would disappear. In the flag complexes of the reconstructions, it was not possible to compute more than the zeroth and top nonzero Betti numbers, as lower dimensions were computationally too expensive (Section 4.2.2). We could easily compute all Betti numbers for the

The Betti number computations showed that there are cavities of dimension 5 (cavities completely enclosed by 5-simplices/6-neuron directed cliques) in all seven instances of each of the reconstructions (Bio1-Bio5, Figure _{5} and the Euler characteristic together captures enough of the structure of the flag complex of a reconstruction to reveal subtle differences in their connectivity arising from the underlying biological data (Figure

Thus far we have shown that the structural network guides the emergence of correlated activity. To determine whether this correlated activity is sufficiently organized to bind neurons together to form active cliques and to bind cliques together to form active cavities out of the structural graph, we represented the spiking activity during a simulation as a time series of sub-graphs for which we computed the corresponding directed flag complexes. Each sub-graph in this series comprises the same nodes (neurons) as the reconstruction, but only a subset of the edges (synaptic connections), which are considered _{1} and the postsynaptic neuron spikes within a time Δ_{2} after the presynaptic spike (Figure _{1}, we obtain a time series of _{1} = 5 ms, Δ_{2} = 10 ms) than would be expected based on the number of edges alone (Figure

The nine stimuli generated different spatio-temporal responses and different numbers of active edges (Figure _{1}) against the number of those of dimension 3 (β_{3}) (the highest dimension in which cavities consistently occur), the trajectory over the course of ~100 ms (Figure _{1} began while β_{3} was still increasing and continued until β_{3} reached its peak, indicating that higher-dimensional relationships between directed simplices continued to be formed by correlated activity as the lower dimensional relationships subside.

_{1}, β_{3}, and Euler characteristic of the time series of TR graphs in response to the stimulus patterns shown in Figure _{1} against β_{3} for three of the stimuli. Shading of colors indicates Gaussian profiles at each time step with means and standard deviations interpolated from 30 repetitions of each stimulus.

Different stimuli led to Betti number trajectories of different amplitudes, where higher degrees of synchrony in the thalamic input produced higher amplitudes. The trajectories all followed a similar progression of cavity formation toward a peak level of functional organization followed by relatively rapid disintegration. The center of the projection of each trajectory onto the β_{1}-axis (its β_{1}-center) was approximately the same. Together, these characteristics of the trajectories reveal a stereotypical evolution of cliques and cavities in response to stimuli. These observations are consistent with experimentally recorded

To determine the neurons involved in this robust evolution of functional organization, we recorded the mean levels of spiking activity at different spatial locations within the microcircuit for one exemplary stimulus (Figure _{1} peaks, consistent with the finding that most directed simplices are in these layers. The transition from increasing β_{1} to increasing β_{3} coincided with the spread of the upper activity zone deeper into layer 5 and the top of layer 6, consistent with the presence of the highest dimensional directed simplices in these layers. The bottom activity zone also continued moving deeper, until it eventually subsided. As the top activity zone reached the bottom of layer 5, β_{3} attained its peak. The zones of activity at the peaks of β_{1} and β_{3} are highly complementary: zones active at the peak of β_{1} were generally inactive at the peak of β_{3} and vice versa. The activity zone then remained in layer 5 until the cavities collapsed.

Finally, we applied the same stimulus to the reconstructions based on variations in the underlying biological data (see Figure _{1}-centers. In some cases, we observed reverberant trajectories that also followed a similar sequence of cavity formation, though smaller in amplitude. The general sequence of cavity formation and disintegration, however, appears to be stereotypic across stimuli and individuals.

This study provides a simple, powerful, parameter-free, and unambiguous mathematical framework for relating the activity of a neural network to its underlying structure, both locally (in terms of simplices) and globally (in terms of cavities formed by these simplices). Using this framework revealed an intricate topology of synaptic connectivity containing an abundance of cliques of neurons and of cavities binding the cliques together. The study also provides novel insight into how correlated activity emerges in the network and how the network responds to stimuli.

Such a vast number and variety of directed cliques and cavities had not been observed before in any neural network. The numbers of high-dimensional cliques and cavities found in the reconstruction are also far higher than in null models, even in those closely resembling the biology-based reconstructed microcircuit, but with some of the biological constraints released. We verified the existence of high-dimensional directed simplices in actual neocortical tissue. We further found similar structures in a nervous system as phylogenetically different as that of the worm

We showed that the spike correlation of a pair of neurons strongly increases with the number and dimension of the cliques they belong to and that it even depends on their specific position in a directed clique. In particular, spike correlation increases with proximity of the pair of neurons to the sink of a directed clique, as the degree of shared input increases. These observations indicate that the emergence of correlated activity mirrors the topological complexity of the network. While previous studies have found a similar link for motifs built from 2-dimensional simplices (Pajevic and Plenz,

Topological metrics reflecting relationships among the cliques revealed biological differences in the connectivity of reconstructed microcircuits. The same topological metrics applied to time-series of transmission-response sub-graphs revealed a sequence of cavity formation and disintegration in response to stimuli, consistent across different stimuli and individual microcircuits. The size of the trajectory was determined by the degree of synchronous input and the biological parameters of the microcircuit, while its location depended mainly on the biological parameters. Neuronal activity is therefore organized not only within and by directed cliques, but also by highly structured relationships between directed cliques, consistent with a recent hypothesis concerning the relationship between structure and function (Luczak et al.,

The higher degree of topological complexity of the reconstruction compared to any of the null models was found to depend on the morphological detail of neurons, suggesting that the local statistics of branching of the dendrites and axons is a crucial factor in forming directed cliques and cavities, though the exact mechanism by which this occurs remains to be determined (but see Stepanyants and Chklovskii,

The digital reconstruction does not take into account intracortical connections beyond the microcircuit. The increase in correlations between neurons with the number of cliques to which they belong should be unaffected when these connections are taken into account because the overall correlation between neurons saturates already for a microcircuit of the size considered in this study, as we have previously shown (Markram et al.,

In conclusion, this study suggests that neocortical microcircuits process information through a stereotypical progression of clique and cavity formation and disintegration, consistent with a recent hypothesis of common strategies for information processing across the neocortex (Harris and Shepherd,

Specializing basic concepts of algebraic topology, we have formulated precise definitions of cliques (

Network where each edge has a |
4.1.1 | |

Clique of all-to-all connected nodes | 4.1.2 | |

Simplex in a directed graph, with a |
4.1.3 | |

The node that is only a source of edges in a directed simplex | 4.1.3 | |

The node that is only a target of edges in a directed simplex | 4.1.3 | |

Obtained by leaving out one or more nodes of a simplex | 4.1.2 | |

A collection of simplices “glued” together along common faces | 4.1.2 | |

Not a face of any larger simplex | 4.1.2 | |

Formalized, intuitive measure of directionality in a graph | 4.1.4 | |

Description of a graph in terms of the number of |
4.1.5.1 | |

Alternating sum of number of simplices | 4.1.5.2 |

A _{1}, τ_{2}): _{1}, _{2}) is taken to be from τ_{1}(_{1}, the _{2}(_{2}, the

There are no (self-) loops in the graph (i.e., for each _{1}, _{2}), then _{1} ≠ _{2}).

For any pair of vertices (_{1}, _{2}), there is at most one edge directed from _{1} to _{2} (i.e., the function τ is injective).

Notice that a directed graph may contain pairs of vertices that are reciprocally connected, i.e., there may exist edges _{1}, _{2}) and

A vertex _{1}(_{2}(_{1}, …, _{n}) such that for all 1 ≤ _{k} is the source of _{k+1}, i.e., τ_{2}(_{k}) = τ_{1}(_{k+1}) (Figure _{1}, …, _{n}) is _{n} is the source of _{1}, i.e., τ_{2}(_{n}) = τ_{1}(_{1}), then (_{1}, …, _{n}) is an

A directed graph is said to be

An

The elements σ of a simplicial complex _{0}, …, _{n}) is a face τ = (_{0}, …, _{m}) for some ^{th} ^{i} obtained from σ by removing the vertex _{n − i}. A simplex that is not a face of any other simplex is said to be

A simplicial complex gives rise to a topological space by

If

Directed graphs give rise to directed simplicial complexes in a natural way. The directed simplicial complex associated to a directed graph _{0}, …, _{n}), of vertices such that for each 0 ≤ _{i} to _{j}. The vertex _{0} is called the _{0}, …, _{n}), as there is an edge directed from _{0} to _{i} for all 0 < _{n} is called the _{0}, …, _{n}), as there is an edge directed from _{i} to _{n} for all 0 ≤

Notice that because of the assumptions on τ, an _{0}, …, _{n}), but not by the underlying set of vertices. For instance (_{1}, _{2}, _{3}) and (_{2}, _{1}, _{3}) are distinct 2-simplices with the same set of vertices.

We give a mathematical definition of the notion of directionality in directed graphs, and prove that directed simplices are fully connected directed graphs with maximal directionality. Let

Note that for any finite graph

Let

Let 𝔽_{2} denote the field of two elements. Let _{n} is the 𝔽_{2}-vector space whose basis elements are the _{n} are formal sums of

For each

specified by ^{i} is the _{n} on the basis, one then extends it linearly to the entire vector space _{n}. The _{2}-vector space dimension of its

for

For all _{n + 1}) ⊆ Ker(∂_{n}) ⊆ _{n}, and thus the definition of homology makes sense.

Computing the Betti numbers of a simplicial complex is conceptually very easy. Let _{n} as a (_{n} with entries in 𝔽_{2}, then one can easily compute its _{n}), and its _{n}), which are the 𝔽_{2}-dimensions of the null-space and the column space of _{n}, respectively. The

Since Im(∂_{n + 1}) ⊆ Ker(∂_{n}) for all _{n} gives an indication of the number of “

If

There is a well-known, close relationship between Euler characterstic and Betti numbers (Munkres,

To obtain the simplices, Betti numbers and Euler characteristic of a directed graph, we first generate the directed flag complex associated to the graph. Our algorithm encodes a directed graph and its flag complex as a

Betti numbers and Euler characteristic are computed from the directed flag complexes. All homology computations carried out for this paper were made with 𝔽_{2} coefficients, using the boundary matrix reduced by an algorithm from the PHAT library (Bauer et al.,

The complexity of computing the _{2} or β_{4} must be nonzero, and it is highly likely the β_{k} is nonzero for all

Analyses of connectivity and simulations of electrical activity are based on a previously published model of neocortical microcircuitry and related methods (Markram et al.,

Additional control models of connectivity were constructed by removing different biological constraints on connectivity. We created three types of random matrices of sizes and connection probabilities identical to the connectivity matrices of the reconstructed microcircuits.

An empty square connection matrix of the same size as the connection matrix of the reconstruction was instantiated and then randomly selected off-diagonal entries were activated. Specifically, entries were randomly selected with equal probabilities until the same number of entries as in the reconstruction were active. The directed graph corresponding to such a matrix is the directed analog of an Erdős-Rényi random graph (Erdos and Rényi,

A square connection matrix was generated based on the existence of spatial appositions between neurons in the reconstruction, i.e., instances where the axon of one neuron is within 1 μ

The connection matrix of a reconstructed microcircuit was split into 55^{2} submatrices based on the morphological types of pre- and postsynaptic neurons. Each submatrix was then randomized by shuffling its connections as follows. Connections in a sub-matrix were first grouped into bins according to the distance between the somata of their pre- and postsynaptic cells. Next, for each connection a new postsynaptic target was randomly selected from the same distance bin. We selected a distance bin size of 75μ

Connectivity between layer 5 thick-tufted pyramidal cells was analyzed using multiple somatic whole-cell recordings (6–12 cells simultaneously) on 300 μ

In order to obtain

We analyzed part of the

We performed simulations of neuronal electrical activity during stimulation with spatio-temporal patterns of thalamic input at the

We used spike trains of 42 VPM neurons extracted from extracellular recordings of the response to texture-induced whisker motion in anesthetized rats, with up to nine cells in the same barreloid recorded simultaneously (Bale et al.,

We constructed postsynaptic time histograms (PSTHs) for each neuron for each stimulus, using the mean response to 30 trials of 5 s of thalamic stimulation (with bin size of 25 ms; for additional control, bin sizes of 10, 50, 100, 250, and 500 ms were also used). We then computed the normalized covariance matrix of the PSTHs of all neurons

where _{ij} is the covariance of the PSTHs of neurons

The temporal sequence of transmission-response matrices associated to a simulation of neuronal activity of duration

where the _{1}, (_{1} + Δ_{2}], and where _{1}. The (

The (

There is some

There is some _{2} interval.

In other words, a non-zero entry in a transmission-response matrix denotes a presynaptic spike, closely followed by a postsynaptic spike, maximizing the possibility of a causal relationship between the spikes. Based on firing data from spontaneous activity in the reconstructed microcircuit, we optimized Δ_{i} such that the resulting transmission-response matrices best reflect the actual sucessful transmission of signals between the neurons in the microcircuit (see _{1} = 5 and Δ_{2} = 10 ms were used throughout the study.

Analysis of the model and simulations was performed on a Linux computing-cluster using Python 2.7, including the numpy and scipy libraries (Jones et al.,

HM and RL developed and initially conceived the study over 10 years of discussions. HM, RL, and KH conceived and directed the final study. KH and RL directed the applicability of concepts in algebraic topology to neuroscience. HM directed the relevance of algebraic topology in neuroscience. The Blue Brain Project team reconstructed the microcircuit and developed the capability to simulate the activity. MN performed the simulations. MN, MR, and PD generated the directed flag complexes from the connection matrices for analysis. KH and RL developed the theory for directed cliques and directed simplicial complexes. MR and RL developed the definition of directionality within motifs and directed cliques. MR developed the definition for transmission response matrices. PD developed the code to isolate simplices and directed simplices and performed initial computations. MS performed topological and statistical analyses on the flag complexes and on the

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

This work was supported by funding from the ETH Domain for the Blue Brain Project and the Laboratory of Neural Microcircuitry. The Blue Brain Project's IBM BlueGene/Q system, BlueBrain IV, is funded by the ETH Board and hosted at the Swiss National Supercomputing Center (CSCS). MS was supported by the NCCR Synapsy grant of the Swiss National Science Foundation. Partial support for PD was provided by the GUDHI project, supported by an Advanced Investigator Grant of the European Research Council and hosted by INRIA. We thank Eilif Muller for providing input on the analysis, Magdalena Kedziorek for help with proving maximality in directed cliques, Gard Spreemann for help with the analysis of the

The Supplementary Material for this article can be found online at: