Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Poster Session C: Friday, August 15, 2:00 – 5:00 pm, de Brug & E‑Hall

Brain-Like Pathways Form in Models With Heterogeneous Experts

Jack Cook¹, Danyal Akarca², Rui Ponte Costa¹, Jascha Achterberg¹; ¹University of Oxford, ²Imperial College London

Presenter: Jack Cook

The brain is made up of a vast set of heterogeneous brain regions that organize themselves into sub-networks and processing pathways to respond to task demands. Examples of such pathways can be seen in the ventral and dorsal visual streams, the Multiple-Demand Network during task execution, and the interaction between cortical and subcortical networks during learning. In this work we ask how do these processing pathways develop from a set of heterogeneous brain regions. Do regions automatically group into systems or are additional priors required? We study this by using neural networks, specifically by extending the Mixture-of-Expert architecture. We show that heterogeneous regions do not automatically form processing pathways by themselves. Training with a processing-complexity routing cost, when scaled based on task performance, results in the development of replicable processing pathways. When comparing our model to the brain, we observe that these pathways match how the brain utilizes different systems to learn and execute tasks of varying task complexities. Our findings establish specific biases that may underlie the formation of processing pathways observed in the brain. At the same time, our model allows us to conduct fine-grained analyses of how sets of pathways interact during problem solving across domains of neuroscience.

Topic Area: Brain Networks & Neural Dynamics

Extended Abstract: Full Text PDF