Configurable character-level transformer training suite with built-in mechanistic interpretability toolkit — scale to 150M+ parameters and beyond, no ceilings, only hardware limits. Inspect attention weights, hidden states, and head specialisation across all layers. Documented circuit findings included.
nlp deep-learning pytorch transformer gpt language-model attention-mechanism circuit-analysis interpretability character-level-language-model attention-visualization attention-heads transformer-interpretability mechanistic-interpretability residual-stream hidden-state-analysis
-
Updated
Jun 5, 2026 - Jupyter Notebook