Applications of information geometry to spiking neural network behavior

Published 12 May 2023 in q-bio.NC and cond-mat.dis-nn | (2305.07482v1)

Abstract: The space of possible behaviors complex biological systems may exhibit is unimaginably vast, and these systems often appear to be stochastic, whether due to variable noisy environmental inputs or intrinsically generated chaos. The brain is a prominent example of a biological system with complex behaviors. The number of possible patterns of spikes emitted by a local brain circuit is combinatorially large, though the brain may not make use of all of them. Understanding which of these possible patterns are actually used by the brain, and how those sets of patterns change as properties of neural circuitry change is a major goal in neuroscience. Recently, tools from information geometry have been used to study embeddings of probabilistic models onto a hierarchy of model manifolds that encode how model behaviors change as a function of their parameters, giving a quantitative notion of "distances" between model behaviors. We apply this method to a network model of excitatory and inhibitory neural populations to understand how the competition between membrane and synaptic response timescales shapes the network's information geometry. The hyperbolic embedding allows us to identify the statistical parameters to which the model behavior is most sensitive, and demonstrate how the ranking of these coordinates changes with the balance of excitation and inhibition in the network.

Abstract PDF Upgrade to Chat

Authors (2)

Summary

The paper identifies that embedding neural activity in hyperbolic space reveals hierarchical sensitivities in network behavior.
It employs a nonlinear Hawkes process and Gaussian approximation to simplify complex spiking dynamics for analysis.
Empirical simulations confirm that variations in excitation-inhibition balance notably shift the network's effective dimensionality.

Applications of Information Geometry to Spiking Neural Network Behavior

Understanding the vast behavioral space of neural circuits is essential for advancing in computational neuroscience. This paper applies tools from information geometry to explore the high-dimensional complexities of spiking neural networks, aiming to quantify the effects of parameter changes on network behavior. By embedding probabilistic models of neural activity into a hyperbolic space, researchers can glean insights into the structure of these behaviors and their sensitivity to certain parameters.

Model Overview

Nonlinear Hawkes Process

The research begins with modeling spiking dynamics using a nonlinear Hawkes process. Each neuron's membrane potential evolves according to input currents and the history of interactions with other neurons, leading to a Poisson-like spike train. The membrane dynamics follow:

$\frac{d V_i}{dt} = -\tau_m^{-1}(V_i-\varepsilon_i) + I_i + \tau_s^{-1}(\mu_{\text{ext}} - J_{\text{self}} \dot{n}_i(t) + \sum_{j=1}^n w_{ij} \dot{n}_j(t))$

This complex model is simplified under a Gaussian approximation, reducing the computation to a population-level description.

Population-Averaged Model

The Gaussian approximation leads to reduced dynamics:

Figure 1: Network model (A) A graphical representation of the network architecture being studied. (B) An example raster plot generated from an extended network of spiking neurons.

In this approximation, the membrane potentials of neuron populations evolve according to:

$d\mathbf{V} = \mathbf{A} (\mathbf{V}^{\text{mf}} - \mathbf{V}) dt + \mathbf{\Sigma} d\mathbf{W}_t$

where $\mathbf{A}$ reflects the drift dynamics, and $\mathbf{\Sigma}$ captures the stochastic influences.

Information Geometric Analysis

isKL Embedding

The intrinsic dimensional structure of neural activity is explored using the isKL embedding technique, mapping probabilistic models onto a Minkowski-like space. This method leverages the symmetric KL-divergence as an effective distance between models with different parameter settings:

Figure 2: Visualizations of the isKL embedding coordinate functions.

Embedding coordinates are derived from exponential family models, allowing analysis of network sensitivity to parameter changes.

Hierarchical Sensitivities

The resulting behavioral manifolds exhibit hierarchical sensitivities. In such structures, certain parameter combinations significantly alter network output, while others contribute to minor adjustments.

Figure 3: Largest manifold projections indicating stiff directions dominating network behavior.

These hierarchical structures are found to shift with excitation-inhibition (E/I) balance, underscoring the dynamic nature of neural configurations.

Empirical Verification

(Later sections detail how these theoretical predictions were verified against full-network simulations. Spiking dynamics were simulated across various timescale settings, confirming the Gaussian approximations.)

Figure 4: Membrane potential dynamics of spiking network models.

Behavioral Space Projections

As E/I ratios change, the manifold's projection hierarchy adapts, revealing how structural properties emerge from the embedded dimensions:

Figure 5: Coordinate evolution as E/I balance is tuned.

Understanding these embeddings helps pinpoint which structural parameters chiefly influence network behavior, aiding in the design of targeted interventions for neurological applications.

Conclusion

The paper systematically applies information geometry to unpack the complex dynamics of spiking neural networks, revealing hierarchical sensitivities in parameter spaces. This approach provides a novel lens for viewing neural function, opening pathways for deeper insights into both normal and pathological states. The embedded analyses serve as robust predictors of network behavior, potentially guiding the development of therapies for neurodisorders by manipulating cardinal features of network structure. Future work should extend these models to dynamic tasks, assessing temporal evolutions in neural coding.