Community Event

Thursday, August 14, 4:15 - 6:00 pm, Room TBA

Representational Alignment (Re^3-Align Collaborative Hackathon)

Brian Cheung, Dota Tianai Dong, Erin Grant, Ilia Sucholutsky, Lukas Muttenthaler, Siddharth Suresh

Abstract

Both natural and artificial intelligences form representations of the world that they use to reason, make decisions, and communicate. But how do we best compare and align these representations? There have been numerous debates around the measures used to quantify representational similarity. As of now, there is little consensus on which metric is best aligned(!) with the goal of identifying similarity between systems. This community event takes a hands-on approach to this challenge through a collaborative hackathon that centers on a decades-long debate: How universal vs. variable are the representations that intelligence systems, biological and artificial, form about the world? This debate has been the target of much recent research in representation learning in machine learning, and is also receiving substantial new attention from neuroscience and cognitive science. During the hackathon, Blue Teams will work to show model universality by finding (or creating) large populations of heterogeneous models that exhibit a high degree of representational alignment. Red Teams will highlight model variability by identifying differences in representations among homogeneous populations of models that are expected to align. The event will open with an interactive, hands-on tutorial on evaluating representational similarity, and will end with summative presentations from the winning teams and an engaging panel discussion with invited speakers.

Session Plan

This collaborative hackathon has two primary objectives: (1) to increase the reproducibility of research in representational alignment to establish shared knowledge through participation in the hackathon, and (2) to facilitate open discussion around identifying the most useful ways of measuring and controlling representational similarity, via presentations and panel discussion. The event begins with an interactive tutorial where organizers will provide starter code and demonstrate essential techniques—extracting model activations, identifying the similarities across different models, and analyzing stimuli that reveal meaningful differences between model representations. Next, the winning teams from the previous hackathon stage will showcase their methods and findings. After these presentations, participants will form new teams for the upcoming hackathon phase. The event concludes with an expert panel exploring challenges and future directions in representational alignment.