Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Poster Session B: Wednesday, August 13, 1:00 – 4:00 pm, de Brug & E‑Hall

TeDFA-$\delta$: Temporal integration in deep spiking networks trained with feedback alignment improves policy learning

Jorin Overwiening¹, M Ganesh Kumar¹, Haim Sompolinsky¹; ¹Harvard University

Presenter: Jorin Overwiening

Limitations in deep spiking reinforcement learning models hinder our understanding of how biological systems learn control policies. We address this by developing a biologically plausible deep reinforcement learning agent (TeDFA-$\delta$) that combines spiking neurons with local Tempotron learning and global Direct Feedback Alignment and Temporal Difference error optimization. Despite using a suboptimal learning rule, TeDFA-$\delta$ outperforms backpropagation-trained MLPs on cartpole, acrobot, and dynamic bandit tasks. This improvement stems from temporal integration of states in spiking neurons rather than the learning algorithm itself, based on ablation studies. The network develops structured spatiotemporal representations where policy and value information coexist, with optimal performance at intermediate membrane time constants ($\tau \ll T$). Our results demonstrate that biological systems may compensate for imperfect credit assignment through temporal dynamics, suggesting neural representations outweigh learning rule optimality for control tasks. This framework enables new studies of biological learning while advancing neuromorphic computing.

Topic Area: Brain Networks & Neural Dynamics

Extended Abstract: Full Text PDF