MH-FLOCKE MH-FLOCKE
HomeDocsGitHubBlogPaperYouTubeReddit𝕏

SNN Controller

Spiking Neural Network — Izhikevich Neurons

The core of MH-FLOCKE is a spiking neural network (SNN) with approximately 4,500 neurons organized in biologically structured populations. Unlike deep learning networks that use continuous activations and backpropagation, this SNN uses discrete spike events and reward-modulated plasticity.

Neuron Model

Each neuron follows the Izhikevich (2003) model — a computationally efficient approximation of Hodgkin-Huxley dynamics that can reproduce 20+ firing patterns observed in cortical neurons:

dV/dt = 0.04V² + 5V + 140 - u + I
du/dt = a(bV - u)

if V ≥ V_threshold:
    V ← c (reset)
    u ← u + d (recovery)
    spike = True

Where V is the membrane potential, u is the recovery variable, I is the input current (from synapses + external), and a, b, c, d are parameters that determine the firing pattern.

Population Architecture

The SNN is organized following the cerebellar circuit (Marr 1969, Albus 1971):

  • Mossy Fibers (MF) — Input population. Sensor values encoded via Gaussian population coding (8 neurons per channel). Includes 2 vision channels for target heading and distance.
  • Granule Cells (GrC, ~4000) — Expansion layer. Each GrC receives exactly 4 MF inputs (biologically: 4 dendrites). Creates sparse, high-dimensional representation. τ_mem = 5ms.
  • Golgi Cells (GoC, ~200) — Inhibitory interneurons. Receive GrC excitation, provide feedback inhibition to enforce sparseness. τ_mem = 20ms.
  • Purkinje Cells (PkC) — 2 per actuator (push/pull). Learn via LTD driven by climbing fiber errors. τ_mem = 15ms.
  • Deep Cerebellar Nuclei (DCN) — 2 per actuator. Motor correction output. τ_mem = 10ms.
  • Output (legacy) — Direct GrC→Output path for SNN motor commands. Population voting (push/pull pairs).

Neuromodulators

Four neuromodulatory systems modulate SNN dynamics globally, analogous to brainstem nuclei:

  • Dopamine (DA) — Reward prediction. Modulates R-STDP learning rate. High DA = reinforce current behavior. Ref: Schultz 1997.
  • Serotonin (5-HT) — Emotional valence. Modulates impulsivity. High 5-HT = patience, low = erratic. Ref: Doya 2002.
  • Norepinephrine (NE) — Arousal and exploration. High NE = explore, low = exploit. Modulates membrane time constants. Ref: Aston-Jones & Cohen 2005.
  • Acetylcholine (ACh) — Attention and learning rate. High ACh = boost R-STDP lr. Active during novel situations. Ref: Hasselmo 2006.

Key Parameters

n_neurons:           ~4,500 (total, varies by morphology)
connectivity:        Structured (MF→GrC: 4 per GrC, not random)
v_threshold:         0.5 (GrC/GoC), 1.0 (PkC/DCN), 0.4 (Output)
stdp_lr:             0.001 (base, modulated by ACh)
eligibility_decay:   0.3 (per apply_rstdp call)
snn_substeps:        6 per creature step (2 propagation cycles)
tonic_current:       0.015 (hidden baseline)

References

  • Izhikevich, E.M. (2003). Simple model of spiking neurons. IEEE Trans. Neural Networks 14(6), 1569-1572
  • Maass, W. (2002). Real-time computing without stable states. Neural Computation
  • Schultz, W. (1997). A neural substrate of prediction and reward. Science
  • Doya, K. (2002). Metalearning and neuromodulation. Neural Networks

API Reference

SNNController(config: SNNConfig)

Core spiking neural network with LIF-LTC neurons, sparse connectivity, R-STDP, neuromodulation, and homeostatic plasticity.

step(external_input: Tensor = None) → Tensor[bool]

One timestep (1ms). Sparse matmul for synaptic input, LIF-LTC membrane update, spike generation, eligibility trace update, homeostatic plasticity. Returns boolean spike tensor [n_neurons].

apply_rstdp(reward_signal: float, prediction_error: float = 0.0)

Weight update: Δw = lr × (0.1×reward + 0.9×(-PE)) × eligibility. ACh boosts lr. Clamped ±0.05/step. Protected populations excluded. E/I sign preserved (Dale’s law).

set_neuromodulator(modulator: str, level: float)

Set global level (0–1). Keys: ‘ne’ (exploration/tau↓), ‘5ht’ (patience/tau↑), ‘ach’ (learning↑), ‘da’ (reward).

get_neuromodulator_tau() → Tensor

Per-neuron membrane time constant modulated by NE and 5-HT. Protected populations keep base tau. Returns [n_neurons].

define_population(name: str, neuron_ids: Tensor)

Register a named population (e.g. ‘granule_cells’, ‘purkinje_cells’).

connect_populations(source: str, target: str, prob: float, weight_range: tuple)

Random sparse connections between populations with given probability and weight range.

get_state() → dict

Returns: firing_rates, mean_potential, e_i_ratio, n_synapses, neuromod_levels, tau_distribution, thresholds_mean, astro_active_clusters.

SNNConfig (key parameters)

n_neurons:            ~4,500 (Go2), 500k default
n_excitatory_ratio:   0.8
tau_base:             20.0 ms (per-population override)
v_threshold:          1.0 (per-population override)
stdp_lr:              0.01 (base, ACh-modulated)
homeostatic_interval: 1000 steps
target_firing_rate:   0.05 (5%)
device:               'cuda' or 'cpu'