Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Poster Session A: Tuesday, August 12, 1:30 – 4:30 pm, de Brug & E‑Hall

Isolating sparse, category-computing circuits in deep neural networks

Jeffery W. Andrade¹, Talia Konkle¹; ¹Harvard University

Presenter: Jeffery W. Andrade

Humans can recognize many object categories: what are the underlying computational routes from retina to category-level representations that enable this capacity? Deep neural networks are now remarkably competent at visual categorization, and as such can serve as a powerful model system for dissecting the hierarchical processing of visual inputs. In this work, we investigate the computations underlying individual category recognition in CNNs: are all unit-to-unit connections across the layers required to categorize an object, or might more dissociable sparse, modular circuits learned in the network support this task? Extending work on CNN circuit extraction (Hamblin, Konkle, & Alvarez, 2023), we have developed a procedure for identifying functional subcircuits within Alexnet that are important for category discrimination. Our algorithm assigns scores to connections based on their estimated contribution to a category unit's activation pattern and prunes the lowest-scored connections up to a chosen circuit substitution accuracy threshold on an extraction imageset. We then evaluate the resulting circuits for function preservation on new images, and analyze the structure of the resulting category circuits. When pruning to an allowed small circuit substitution accuracy decrement, we find surprisingly sparse, substantially faithful circuits with an average circuit sparsity of 45.3% and an average circuit substitution accuracy of 90.9% that of the unpruned network. These results indicate that category-level representations individually depend upon relatively sparse subnetworks, suggesting a semi-modular neural code with significant, structured sharing of circuitry.

Topic Area: Object Recognition & Visual Attention

Extended Abstract: Full Text PDF