What if the brain is a rhizome?

· Thoughts and opinions are my own, albeit cybernetically enhanced.

This post summarizes rhizomatic networks, a series of experiments — seven completed, an eighth underway — investigating whether a recurrent network with no hierarchical organization can perceive and learn using only locally available information. The name borrows from the rhizome, a term for a decentralized structure in which any point can connect to any other1 — an informal but reasonable description of cortical connectivity, which is sparse, recurrent, and small-world in character,2 with feedback pathways as prominent as feedforward ones.34

Most trainable architectures are organized quite differently: as stacks of layers trained by backpropagation, in which activity ascends a hierarchy and error descends it in a separate, globally coordinated pass.5 The question the project set out to answer is whether that structure is necessary. Take a network that has none of it — a small-world graph of simple units, each with its own timescale, no layers, no global controller, no separate backward phase — and determine what it can perceive and whether it can learn, using rules that depend only on quantities local to each node and synapse.

The short version of what seven experiments found: perception on such a substrate works, and is robust for reasons specific to its decentralization. Learning splits into two problems. Credit assignment through the graph’s loops at a moment in time has an exact, cheap, local solution. Credit assignment backward through time does not — its exact form is local in space but heavy in state, its compressed forms fail in identifiable ways, and characterizing that boundary is where the project now sits.

The model

All experiments use variants of a single substrate: 128–1024 scalar nodes on a sparse random graph,6 coupled by a weight matrix \(W\). Each node has a membrane potential \(u\) and a firing rate \(r\), evolving as

\[\tau_m \,\dot{u} = -u + W r + b, \qquad r = \phi\!\left(u + \tau_r\, \dot{u}\right)\]

The first equation is standard leaky integration of graph neighbours, with a per-node time constant \(\tau_m\). The second makes the rate prospective: a node fires according to a first-order extrapolation of its potential, \(\tau_r\) into the future.7 Time constants are heterogeneous across nodes, so the network contains both fast and slow intrinsic dynamics. Designated sensory nodes receive input, designated class nodes are read out, and the remainder are unstructured latent nodes.

Perception on a non-hierarchical substrate

The first experiment presented MNIST digits8 as temporal sequences — a one-column slit scanning across the image — so that recognition requires integrating evidence over time. Three findings were robust across seeds and controls:

Recognition is order-tolerant while the underlying process is path-dependent. Presenting observations in a different order yields the same classification 83–89% of the time, while the final latent state differs by up to 1.7× its own norm. The computation admits many trajectories to the same decision.

Performance does not depend on hub nodes. Removing the most-connected nodes degrades accuracy no more than removing random nodes, whereas layered comparison models are strongly hub-dependent.

Function is decoupled from anatomy. Functional similarity between nodes is uncorrelated with graph distance (Spearman ≈ 0), and structural plasticity rewires the network away from its initial lattice without measurable effect on accuracy.

Two failure modes of local credit assignment

The obstacle to learning on this substrate is credit assignment, and two experiments characterized how the obvious local approaches fail.

The second experiment tested whether propagating credit spatially — letting nodes exchange credit messages over 1–4 hops of the graph — extends the temporal range of local learning. It does not. No radius improved on the zero-hop rule, and a control with random feedback weights9 performed equivalently, indicating the mechanism carried no structure-specific information. What did determine the temporal range was memory: the length of each synapse’s eligibility trace1011 sets the horizon, with the best horizon approximately twice the task delay.

The third experiment evaluated generalized latent equilibrium (GLE), a principled local learning theory in which error signals propagate through the same prospective dynamics as activity.12 On hierarchies it performs as reported. On graphs with cycles it is intrinsically unstable: the error operator has a high-frequency gain of \(\tau_m/\tau_r\) per hop, which is applied once per layer in a hierarchy but compounds without bound around a cycle. A linearized stability analysis predicted every observed divergence, and the regimes stable enough not to diverge carried no usable gradient information. The instability is a property of the operator, not of the discretization.

Spatial credit as a fixed point

The fourth experiment reformulated the problem. Rather than transporting error through the forward dynamics, it asks what the correct credit signal on a cyclic graph is. With \(J\) the single-step Jacobian and \(q\) the error injected at the output nodes, the appropriate error field is the fixed point

\[e = (I - J^\top)^{-1} q = q + J^\top q + (J^\top)^2 q + \cdots\]

The series form is informative: correct credit is a sum over all walks through the graph, cycles included, each weighted by its gain. This is the fixed point computed by the recurrent backpropagation algorithms of the 1980s,13 and the standard backward pass is the acyclic special case.

This fixed point is reachable by a strictly local relaxation. Each node maintains one auxiliary error state \(z\), updated using only messages from immediate neighbours:

\[z \;\leftarrow\; z + \alpha\left[\, D\,q \;-\; z \;+\; \kappa\, D\, W^\top z \,\right], \qquad \Delta W_{ij} \;\propto\; z_i\, r_j\]

where \(D\) is the local activation slope and \(\kappa < 1\) is a loop discount that down-weights long walks. The weight update is a conventional product of presynaptic activity and a postsynaptic error factor.14 On frozen networks this field aligns with the true gradient at cosine ≈ 1 wherever the GLE error field was uncorrelated with it; run online at one relaxation step per dynamics step, it tracks the moving credit target to within 1%; and in training it learns tasks that require routing credit through the cycles, exceeding the best non-learning heuristic where backpropagation-through-time15 trained on the same objective does not. Within a single observation, spatial credit on a loopy graph is, for practical purposes, solved locally.

What the local rule still misses

The fifth experiment pushed the combination — loop-aware field plus eligibility trace — to its limit on the scanned-MNIST task, and located the residual gap precisely.

First, a correction the larger budget forced: the substrate is more capable than earlier runs suggested. Trained longer and at a higher learning rate, BPTT on the identical network reaches 0.87 on the moving slit (0.93–0.94 when the whole image is visible), so earlier, lower figures were budget artifacts, not architecture limits. Against that ceiling, the best local configuration — loop discount κ = 0.7, trace horizon of 8 observations, and supervision only at the end of the sequence, which alone is worth 5 points — reaches 0.77, i.e. 88% of the oracle.

Three negative results sharpened where the missing 10 points live. Topology is not the answer: at matched size, fan-in, and edge count, small-world, Erdős–Rényi, and ring substrates perform identically for both rules — what the local rule needs is sparsity, not any particular wiring statistics, a refinement of the earlier topology results. Capacity is not the answer: scaling the network from 128 to 1024 nodes gives BPTT 12 points and the local rule none. And the trace is not the answer: temporal attribution analysis — asking which input columns each rule’s weight updates actually derive from — shows the informative columns sit in the middle of the digit, BPTT’s credit follows them (center of mass at column 14), while the trace rule’s credit comes from the last third of the sequence (center of mass at column 21), and no trace length fixes this. Longer traces smear credit and lose accuracy. Temporal persistence, it turns out, is not temporal influence.

Temporal influence, computed exactly

The sixth experiment made that distinction precise. On a recurrent substrate, a synapse’s influence on a later output propagates through the graph’s loops, and candidates for tracking it can be ordered by how much of that propagation they see: the presynaptic trace (none), the node’s own self-path, a neighbourhood of radius \(R\), and, in the limit, the exact forward sensitivity of real-time recurrent learning.16 On a causal-chain benchmark built to have known depth, the radius required grows with the depth, exactly as the geometry implies — and on MNIST the finite radii improve monotonically but slowly, because every loop traversal costs a hop.

The limit case settles the question: the exact forward sensitivity, contracted with the same loop-aware field (so the error side stays local and cycle-corrected), reaches 0.868 ± 0.003 versus BPTT’s 0.868 ± 0.015 on the unchanged substrate — with BPTT’s temporal attribution profile. The credit gap of experiments 2 through 5 is entirely a temporal-influence gap. The cost is the catch: exact sensitivity carries one number per (weight, node) pair, roughly \(6 \times 10^5\) values per sample here, which is local in space but not remotely bounded per synapse.

Compression fails, informatively

The seventh experiment asked whether that sensitivity can be compressed into a bounded number of locally propagated channels. Offline, the answer is emphatically yes: the sensitivity is low-rank along the state axis — a rank-1 approximation already captures 84% of its energy, and 32 modes give cosine 0.995 with the exact quantity. Online, every candidate tried fails. Hashed count-sketch channels along the weight axis drown the per-synapse signal in collision noise (4–16× the signal) — except on tasks whose credit is sparse, where they work, a clean diagnostic that MNIST’s credit is dense. Unbiased stochastic approximations (UORO17) have variance two orders of magnitude above the signal at this scale. And projecting the recursion onto a fixed modal basis has no stable closure: the reduced Jacobian is expansive, and once capped, the recursion forgets any component of the sensitivity that leaves the subspace and returns. The outcome is sharp: compressible in principle, unresolved algorithmically — and the failure modes say why each family fails.

The eighth experiment, now running, inverts the question: instead of compressing the sensitivity of a fixed architecture, it constrains the architecture — invariant low-rank memory cores, modular blocks, normalized recurrence — so that the exact sensitivity closes in a bounded number of locally maintained values per synapse by construction.

The loop-gain boundary

Across all seven experiments, one quantity organizes the results: the loop gain \(\rho(D W^\top)\), the spectral radius governing how much a signal grows per traversal of the graph’s cycles.

Where this leaves the question

A non-hierarchical recurrent network perceives robustly precisely because it is decentralized — no critical hubs, no canonical trajectory — and it can learn from purely local rules up to a well-defined limit. Spatial credit through loops is exact and cheap: a one-hop relaxation with a discount. Temporal credit is the honest frontier: its exact form is spatially local but state-heavy, its compressions fail for reasons that are now characterized rather than mysterious, and the open question has narrowed from “can a rhizome learn?” to “which architectures make exact temporal credit closeable in bounded local state?” That is a much better question than the project started with.

The front page of this site runs the project’s final local configuration — 28 sensory nodes, 128 latent nodes on a Watts–Strogatz graph, 10 class nodes, loop-aware field at κ = 0.7 with terminal supervision and an eligibility horizon of 8 observations — live in a WebGPU compute shader, learning as it runs. Green edges carry forward activity (blue where the weight is inhibitory), dark traffic is the plasticity signal, and the panel in the corner shows the current input digit and the network’s running classification.

References

  1. Gilles Deleuze and Félix Guattari, A Thousand Plateaus: Capitalism and Schizophrenia, trans. Brian Massumi (Minneapolis: University of Minnesota Press, 1987). Originally Mille Plateaux (Paris: Éditions de Minuit, 1980). Lending scan at the Internet Archive. ↩

  2. Danielle S. Bassett and Ed Bullmore, “Small-world brain networks,” The Neuroscientist 12, no. 6 (2006): 512–523. doi:10.1177/1073858406293182. ↩

  3. Daniel J. Felleman and David C. Van Essen, “Distributed hierarchical processing in the primate cerebral cortex,” Cerebral Cortex 1, no. 1 (1991): 1–47. doi:10.1093/cercor/1.1.1. ↩

  4. Nikola T. Markov et al., “Cortical high-density counterstream architectures,” Science 342 (2013): 1238406. doi:10.1126/science.1238406. ↩

  5. David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams, “Learning representations by back-propagating errors,” Nature 323 (1986): 533–536. doi:10.1038/323533a0. ↩

  6. Duncan J. Watts and Steven H. Strogatz, “Collective dynamics of ‘small-world’ networks,” Nature 393 (1998): 440–442. doi:10.1038/30918. ↩

  7. Paul Haider et al., “Latent Equilibrium: a unified learning theory for arbitrarily fast computation with arbitrarily slow neurons,” Advances in Neural Information Processing Systems 34 (2021). arXiv:2110.14549. ↩

  8. Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE 86, no. 11 (1998): 2278–2324. doi:10.1109/5.726791. ↩

  9. Timothy P. Lillicrap, Daniel Cownden, Douglas B. Tweed, and Colin J. Akerman, “Random synaptic feedback weights support error backpropagation for deep learning,” Nature Communications 7 (2016): 13276. doi:10.1038/ncomms13276. ↩

  10. Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, 2nd ed. (Cambridge, MA: MIT Press, 2018), ch. 12. incompleteideas.net/book. ↩

  11. Guillaume Bellec et al., “A solution to the learning dilemma for recurrent networks of spiking neurons,” Nature Communications 11 (2020): 3625. doi:10.1038/s41467-020-17236-y. ↩

  12. Benjamin Ellenberger et al., “Backpropagation through space, time, and the brain,” Nature Communications 17 (2026): 66, doi:10.1038/s41467-025-66666-z; arXiv:2403.16933. ↩

  13. Luís B. Almeida, “A learning rule for asynchronous perceptrons with feedback in a combinatorial environment,” in Proceedings of the IEEE First International Conference on Neural Networks (1987), 609–618; Fernando J. Pineda, “Generalization of back-propagation to recurrent neural networks,” Physical Review Letters 59 (1987): 2229–2232. doi:10.1103/PhysRevLett.59.2229. ↩

  14. Donald O. Hebb, The Organization of Behavior: A Neuropsychological Theory (New York: Wiley, 1949); reissued (Mahwah, NJ: Psychology Press, 2002), doi:10.4324/9781410612403; scan of the 1949 edition. ↩

  15. Paul J. Werbos, “Backpropagation through time: what it does and how to do it,” Proceedings of the IEEE 78, no. 10 (1990): 1550–1560. doi:10.1109/5.58337. ↩

  16. Ronald J. Williams and David Zipser, “A learning algorithm for continually running fully recurrent neural networks,” Neural Computation 1, no. 2 (1989): 270–280. doi:10.1162/neco.1989.1.2.270. ↩

  17. Corentin Tallec and Yann Ollivier, “Unbiased Online Recurrent Optimization,” International Conference on Learning Representations (2018). arXiv:1702.05043. ↩