ML engineers
You train or deploy models and want the practical skills to inspect what happens inside them.Learn to inspect the models you build.
About this course
Mechanistic interpretability tries to explain what a neural network computes by reading its internals, not only its outputs. If we can find the features and circuits behind a model’s behaviour, we can check for deception, hidden knowledge, and unsafe goals more directly than behavioural tests allow.
This is a hands-on coding course. You write a transformer yourself, load GPT-2 weights into it, and then use the standard tools of the field to take it apart: activation hooks, linear probes, steering vectors, sparse autoencoders, and circuit analysis. Each unit is a set of ARENA exercises, followed by a small-group session to compare results.
You need to be comfortable with Python and PyTorch, and to know basic linear algebra and how neural networks are trained. You do not need prior interpretability experience.
Who is it for:
Aspiring interpretability researchers
You want to replicate core results yourself before you apply to a fellowship such as MATS, or start your own project.Replicate core results before a fellowship.
Researchers from other sciences
You can code and you know how to run experiments, and you want to apply that to neural networks.Bring your experimental skills to neural networks.
Safety evaluators
You test model behaviour and want white-box methods, such as probes and feature monitoring, to add to black-box evaluations.Add white-box methods to your evaluations.
Curriculum
5 UNITS · ~5H EACHA core path through ARENA’s Transformer Interpretability chapter. ARENA’s further sections, such as OthelloGPT, grokking, and the balanced bracket classifier, are suggested as follow-up work.
Transformers from scratch
Implement every part of a GPT-2 style transformer: embeddings, attention, MLPs, and layer norm. Load the pretrained GPT-2 weights into your own code, then sample text from it.
Introduction to mechanistic interpretability
Learn TransformerLens. Cache and inspect activations, write hooks that intervene in a forward pass, and find important attention heads, including the induction heads behind in-context learning.
Probing and steering
Train linear probes that detect deception in a model playing a bluffing game. Then extract function vectors and steer a model’s behaviour by intervening on its activations.
Superposition and sparse autoencoders
See why models store more features than they have neurons. Use sparse autoencoders to split a language model’s activations into interpretable features, then monitor and steer the model with them.
Circuits
Reverse-engineer the indirect object identification circuit in GPT-2 small with activation patching and path patching. Then trace sparse autoencoder features through the layers to explain a computation end to end.
Course Outcomes
A transformer you wrote yourself
You can explain every tensor in a forward pass, because you implemented each one.
Fluency with the standard tools
You can use TransformerLens and nnsight to cache activations, add hooks, and run interventions.
Probes and steering vectors that work
You can train a linear probe, judge what its accuracy does and does not show, and steer a model with activation interventions.
Working knowledge of sparse autoencoders
You can explain superposition, use sparse autoencoders to find features, and state the known limits of the method.
A circuit you reverse-engineered
You have replicated a published circuit analysis and can apply patching methods to a new behaviour.
A starting point for research
You know the open problems and have the code to begin a small interpretability project of your own.
Upcoming cohorts
INTENSIVE
5h/day · 1 week
Dates to be announcedWe’ll publish the next intensive cohort here.
Common questions
Not sure it’s for you? Come to the first session. If it’s not the right fit, we’ll help you find something that is.