ClimbingBoardGPT uses AI to generate new climbing routes and predict their difficulty — for Tension Board 2 and Kilter Board.
The route generator follows the small causal-transformer recipe from Andrej Karpathy's nanoGPT: train a model to predict the next token, then sample from it autoregressively. The twist here is that the "language" is made of board, angle, grade, and hold-role tokens instead of text.
You give it a board, a wall angle, and a target grade. It gives you a route. You can also paste in a route you already know, and it will guess the grade.
- What is this, exactly?
- What are Tension Board 2 and Kilter Board?
- What can it do?
- Try it
- How it works
- Climbing-board ML $\leftrightarrow$ Linear Algebra Dictionary
- Quantitative results
- Setup
- Run the web demo locally
- CLI demos
- Full training pipeline
- API endpoints (for the webapp)
- Repository layout
- Important caveats
- Background and related work
- License
If you've climbed on a Tension Board 2 (TB2) or a Kilter Board, you know these are standardized training boards with a fixed set of holds. Routes on these boards are described as a list of holds and their roles (start, foot, hand, finish). The holds are identified by placement ID numbers and the route is stored as a short string like p652r5p631r6p322r6p326r7.
This project trains two small AI models on hundreds of thousands of real community-set routes from both boards:
- A route generator — you ask for a V6 at 40° on the Kilter, and the model samples a novel sequence of holds that should produce something around that difficulty.
- A grade predictor — you give it any route (board + angle + holds), and the model estimates the difficulty.
Both models are transformer-based neural networks, the same family of architecture behind large language models. Here the "language" is not English words but climbing-hold tokens: each hold-role combination gets its own symbol, and a route is a short sentence in that language.
The whole thing is small by modern standards (~1.2–1.4M parameters each) and runs on a CPU.
Tension Board 2 (TB2) is an adjustable training wall made by Tension Climbing. It has a fixed set of holds placed in a regular grid. Climbers set and share routes through a companion app; the community has set tens of thousands of problems. We work with the 12x12 ft mirror in this project.
Kilter Board is a similar product from Kilter (Setter Closet). It also has a large library of community-set problems. We work with the 16x12 ft Kilter original board in this project.
Both boards store routes as placement-ID strings. That is what this project trains on.
| Feature | How to use it |
|---|---|
| Generate new routes | Web app or CLI script |
| Predict grade from holds | Web app or CLI script |
| Visualize routes on a board image | CLI script, saved as PNG/SVG |
| Run a local web demo | Docker or uvicorn |
| Retrain from scratch | Four numbered scripts |
The live demo is at cbgpt.pawelsarkowicz.xyz. You can:
- Pick a board (TB2 or Kilter), a wall angle, and a target grade, and click Generate to get a new route drawn on the board image.
- Paste a frames string — the compact route code used by the board apps — into the Predict tab to estimate the grade of any route you already know.
Every route is converted into a short sequence of symbols, one per hold:
<BOS> <BOARD_TB2> <ANGLE_40> <GRADE_V6>
<TB2_p344_start> <TB2_p369_middle> <TB2_p603_finish>
<EOS>
<BOS> and <EOS> mark the start and end. The board, angle, and grade tokens say: "this is a TB2 problem, set at 40 degrees, graded V6." The rest are hold tokens — each one encodes a specific hold and whether it is a start, middle, finish, or foot hold.
Both boards share one vocabulary, so the model can learn patterns from TB2 and Kilter routes together without confusing hold positions from one board with the other.
The grade predictor reads the sequence above (minus the grade token, which it has to guess) and outputs a single number. It is a transformer encoder — roughly the same kind of model used for text classification, just applied to climbing holds instead of words.
It also gets the physical (x, y) board coordinates of each hold as extra input, so it can reason about route geometry: how high the holds are, how far apart they are, whether the route traverses sideways, etc.
Accuracy on the held-out test set: 79% within ±1 V-grade.
The route generator is a small GPT-style model — the same general idea as ChatGPT, but tiny and trained only on climbing routes. You give it the prompt:
<BOS> <BOARD_KILTER> <ANGLE_40> <GRADE_V6>
and it predicts what hold token should come next, then the next, then the next, until it produces an <EOS> token. The result is a novel sequence of holds that the model thinks is a plausible V6 on the Kilter at 40°.
~89% of generated routes pass basic structural checks (has a start hold, has a finish hold, holds exist on the right board, no duplicates).
The single most useful thing to keep in mind: a route is a short sequence of discrete symbols, and almost every operation in this project is a standard linear-algebra operation on the matrices and vectors built from those symbols.
Each pipeline stage (tokenization, encoder training, generator training, evaluation) has its own table below. The project-wide reference, with the notebook(s) where each term appears, follows.
Notebook numbering: 01 Tokenization · 02 Grade prediction (transformer encoder) · 03 Route generation (GPT-style decoder) · 04 Generated-route evaluation.
| Term | Linear-algebra meaning | Notebooks |
|---|---|---|
| Hold | A discrete symbol drawn from a finite placement set — one element of the board's hold-index set | 01 |
| Role | A label (start, middle, finish, foot) attached to a hold; part of the token's identity, not a separate axis | 01 |
| Frames string | Compact serialization |
01 |
| Route | An ordered list of |
01 |
|
Vocabulary |
The finite symbol set, |
01 |
stoi / itos |
The bijection between tokens and basis-vector indices |
01 |
| Special tokens | Distinguished basis vectors ( |
01 |
| Board namespacing | A direct-sum partition |
01 |
| Encode / decode | Convert tokens to basis indices and back: |
01 |
| Canonical ordering | A fixed sort key on holds (role, |
01 |
| Grade tokens | Discrete grade bins GRADE_TO_V
|
01 |
| Train / val / test split | A partition of the row index set into three disjoint subsets, stratified by board |
01 |
| Term | Linear-algebra meaning | Notebooks |
|---|---|---|
|
One-hot map |
|
01, 02, 03 |
|
Token embedding matrix |
Learned linear map |
02, 03 |
| Embedding lookup | The matrix-vector product |
02, 03 |
padding_idx=0 |
Pins column |
02, 03 |
| Position embedding | A second embedding matrix |
02, 03 |
|
Coordinate features |
A fixed matrix |
02 |
| Coordinate normalisation | Affine rescaling |
01, 02 |
|
Coordinate projection |
Learned linear map |
02 |
| Token + position + coord | Three vectors added componentwise in |
02 |
|
Sequence matrix |
Stacked row embeddings: |
02, 03 |
| Term | Linear-algebra meaning | Notebooks |
|---|---|---|
| Self-attention | A learned bilinear form followed by a convex combination: |
02 |
| Query / Key / Value | Three learned linear images of |
02 |
| Multi-head attention | Block-diagonal decomposition of |
02 |
|
Gram matrix |
The |
02 |
| Variance normalisation so the inner products have |
02 | |
| Row-wise softmax | Each row of |
02 |
| Convex combination |
|
02 |
key_padding_mask |
Zeros out rows of the attention matrix that correspond to pad tokens | 02 |
| Pre-norm residual |
|
02 |
| LayerNorm | Affine map subtracting the mean and dividing by the std of each row, then scaling/shifting — projection onto the unit sphere followed by a learned affine transform | 02 |
| Feed-forward block | Two linear maps |
02 |
<CLS> pooling |
Selecting row |
02 |
| Regression head | A learned nonlinear functional |
02 |
| Term | Linear-algebra meaning | Notebooks |
|---|---|---|
| Causal mask | Strictly upper-triangular boolean matrix |
03 |
| Masked attention | Attention restricted to the lower-triangular part of the Gram matrix | 03 |
| Teacher forcing | Training pairs |
03 |
|
Logits |
A vector |
03 |
| Weight tying |
lm_head.weight = token_emb.weight; the output projection reuses |
03 |
| Cross-entropy loss |
|
03 |
| Perplexity |
|
03 |
| Block size | The context window |
03 |
| Term | Linear-algebra meaning | Notebooks |
|---|---|---|
|
Logits vector |
An arbitrary vector in |
03, 04 |
|
Temperature |
Divides |
03, 04 |
| Top-$k$ filter | Masks every logit outside the top |
03, 04 |
forbidden_ids |
Sets specified logits to |
03 |
| Softmax | Map |
03, 04 |
| Multinomial sample | Draw one basis vector |
03, 04 |
| Autoregressive loop | Ancestral sampling: concatenate <EOS>
|
03, 04 |
| Term | Linear-algebra meaning | Notebooks |
|---|---|---|
| MSE loss | Squared Euclidean distance |
02 |
| AdamW | Adaptive-gradient SGD using a diagonal preconditioner (running means of |
02, 03 |
| Weight decay | Decoupled L2 regularisation that shrinks every parameter vector toward |
02, 03 |
| Gradient clipping | Rescale the gradient: |
02, 03 |
|
Learning rate |
Step size along |
02, 03 |
| Batch | A uniform sample of training rows; loss is the mean over the batch | 02, 03 |
| Early stopping | Halt when the validation metric stops improving; restore the parameters with best validation score | 02, 03 |
| Train / val / test | Three disjoint samples of rows used for fitting, hyperparameter selection, and final evaluation | 02, 03 |
| Term | Linear-algebra meaning | Notebooks |
|---|---|---|
| Jaccard similarity |
|
04 |
| Novelty distance |
|
04 |
| Nearest real route | Argmax of Jaccard over the training set — an exhaustive nearest-neighbour query under the Jaccard metric | 04 |
| MAE |
|
04 |
| RMSE |
|
04 |
| Fraction of variance explained: |
04 | |
| Within ±$k$ V-grades |
|
04 |
| Hand-reach distances |
scipy.spatial.distance.pdist(holds[["x","y"]].values) — pairwise Euclidean distances in |
04 |
| Composite ranking score | Learned linear combination of validity, novelty, and grade-consistency features used to rank generated routes | 04 |
These numbers are from the full training run documented by this repository. The notebooks in notebooks/ are self-contained walkthroughs of the pipeline stages. The reported pipeline run was executed on Kaggle; notebooks 01-04 took about 8h 1m 59s total using GPU T4 x2.
In practice: the grade model is usually within one V-grade, and the generator usually makes structurally valid routes, but exact grade control is still imperfect.
| Board | Training routes | Validation | Test |
|---|---|---|---|
| TB2 | 33,719 | 4,430 | 4,447 |
| Kilter | 223,112 | 27,555 | 27,822 |
| Total | 256,831 | 31,985 | 32,269 |
Shared vocabulary:
The model has ~1.17M parameters (scalar entries across all parameter tensors). Early stopping selected epoch 11 (validation MAE
| Metric | Overall | TB2 | Kilter |
|---|---|---|---|
| Exact V-grade | 35.8% | 35.8% | 35.8% |
| Within ±1 V-grade | 79.2% | 79.4% | 79.1% |
| Within ±2 V-grades | 94.9% | 95.5% | 94.8% |
| 0.763 | 0.793 | 0.758 |
The generator has ~1.41M parameters. Best validation perplexity: 24.3.
| Metric | TB2 | Kilter |
|---|---|---|
| Routes evaluated | 200 | 200 |
| Structurally valid | 91.5% | 86.0% |
| Exact requested grade (critic) | 34.5% | 37.0% |
| Within ±1 V-grade (critic) | 73.0% | 79.5% |
| Within ±2 V-grades (critic) | 91.0% | 96.5% |
| Mean novelty (Jaccard distance) | 0.656 | 0.643 |
Requires Python 3.12+.
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -e .uvicorn webapp.app:app --host 127.0.0.1 --port 8055Then open http://127.0.0.1:8055.
The webapp needs these files (generated by training, or copied from a previous run):
models/joint_route_gpt_generator.pth
models/joint_transformer_grade_predictor.pth
data/processed/tokenized/token_metadata.csv
data/processed/tokenized/token_vocab.json
data/processed/tokenized/route_sequences.csv
configs/
images/
docker compose -f docker-compose.webapp.yml up -d --buildBinds to 127.0.0.1:8055.
Use the reverse-proxy compose file when another container, such as Caddy,
owns the public ports and shares an external Docker network with the webapp.
The file defaults that external network to caddy:
docker network create caddy
docker compose -f docker-compose.webapp.proxy.yml up -d --buildThis does not publish a host port. It exposes port 8055 only on the proxy
network. For Caddy, a site block can reverse proxy to the webapp container:
cbgpt.example.com {
reverse_proxy climbingboardgpt-webapp:8055
}Once the trained model checkpoints are in models/, you can run demos from the terminal.
# TB2, 40 degrees, V6, 4 routes
python scripts/demo_generate_tb2.py --angle 40 --grade 6 --n 4
# Kilter
python scripts/demo_generate_kilter.py --angle 40 --grade 6 --n 4
# With custom temperature and top-k sampling
python scripts/demo_generate_and_visualize.py \
--board tb2 --angle 40 --grade 6 --n 4 \
--temperature 0.9 --top-k 50Generated routes are saved to:
outputs/demo_routes/<board>/angle_<angle>/V<grade>/
├── generated_routes.csv
├── generated_route_001.png
├── generated_route_001.svg
...
What does temperature do?
| Temperature | Effect |
|---|---|
0.3–0.6 |
Conservative — picks safer, more common moves |
0.9 |
Balanced default |
1.0 |
Samples directly from learned probabilities |
1.1–1.3 |
More creative — can produce weirder routes |
What does top-k do?
| Top-k | Effect |
|---|---|
1 |
Greedy — always picks the single most likely token |
5–15 |
Very focused — limited vocabulary per step |
50 |
Balanced default |
200–500 |
Less restricted — more variety per step |
The two parameters work at different stages of the same step. At each position in the sequence, top-k first discards every token outside the k most likely candidates. Temperature then rescales the probabilities of those remaining candidates before drawing from them. Setting top_k=1 makes temperature irrelevant — there is only one candidate left. With a larger top-k, temperature controls how evenly spread the draw is: low temperature concentrates weight on the top few candidates, high temperature flattens the distribution toward equal probability across all k.
In short: top-k controls the breadth of what is considered; temperature controls how adventurously the model picks within that breadth. The defaults (top_k=50, temperature=0.9) aim for a balance between variety and structural coherence.
# TB2
python scripts/demo_predict_tb2.py \
--angle 40 --frames 'p652r5p631r6p322r6p326r7'
# Kilter
python scripts/demo_predict_kilter.py \
--angle 40 --frames 'p1127r12p1196r13p1216r13p1388r14'Example output:
Board: Tension Board 2 Mirror (tb2)
Angle: 40°
Frames: p652r5p631r6p322r6p326r7
Predicted: V6
Additional flags: --json for machine-readable output, --visualize to save a board image, --show-tokens to inspect the token sequence.
To train from scratch you need the raw board databases at:
data/raw/tb2.db
data/raw/kilter.db
These can be downloaded with the BoardLib CLI — the commands are recorded in configs/tb2.json and configs/kilter.json.
Then run the four scripts in order:
python scripts/01_tokenize_routes.py --boards tb2,kilter
python scripts/02_train_grade_predictor.py
python scripts/03_train_route_generator.py
python scripts/04_evaluate_generated_routes.pyThis produces trained checkpoints in models/ and evaluation outputs in data/processed/.
To verify the pipeline runs end-to-end without retraining the real models, once the raw board databases are in data/raw/:
python scripts/01_tokenize_routes.py \
--out-dir /tmp/cbgpt_smoke/tokenized \
--max-routes-per-board 20
python scripts/02_train_grade_predictor.py \
--tokenized-dir /tmp/cbgpt_smoke/tokenized \
--out-dir /tmp/cbgpt_smoke/grade_prediction \
--model-dir /tmp/cbgpt_smoke/models \
--smoke-test
python scripts/03_train_route_generator.py \
--tokenized-dir /tmp/cbgpt_smoke/tokenized \
--out-dir /tmp/cbgpt_smoke/generation \
--model-dir /tmp/cbgpt_smoke/models \
--smoke-test \
--generate-angles 40 \
--generate-grades 6
python scripts/04_evaluate_generated_routes.py \
--tokenized-dir /tmp/cbgpt_smoke/tokenized \
--generated-dir /tmp/cbgpt_smoke/generation \
--out-dir /tmp/cbgpt_smoke/evaluation \
--grade-model-path /tmp/cbgpt_smoke/models/joint_transformer_grade_predictor.pth \
--device cpuThe numbers from this run are meaningless — it only checks that the code runs.
GET /api/health
GET /api/boards
POST /api/generate
POST /api/predict
Example generation payload:
{
"board": "tb2",
"angle": 40,
"grade": 6,
"temperature": 0.9,
"top_k": 50,
"max_new_tokens": 40
}Example prediction payload:
{
"board": "kilter",
"angle": 40,
"frames": "p1127r12p1196r13p1216r13p1388r14"
}ClimbingBoardGPT/
├── configs/ Board-specific config files (role IDs, angle ranges, etc.)
├── data/
│ ├── raw/ Raw SQLite databases (not in git)
│ └── processed/ Tokenized data and training outputs (not in git)
├── images/ Board background images
├── models/ Trained model checkpoints (not in git)
├── notebooks/ Executed Jupyter notebooks documenting each pipeline step
├── scripts/ Training scripts (01–04) and CLI demo scripts
├── src/climbingboardgpt/ Importable package — models, tokenization, inference, etc.
├── tests/ Unit tests
├── webapp/ FastAPI server + browser-side SVG route builder
├── docker-compose.webapp.yml
├── docker-compose.webapp.proxy.yml
├── requirements.txt
└── pyproject.toml
The main package modules:
| Module | What it does |
|---|---|
config.py |
Loads board JSON configs and role mappings |
data.py |
Reads from the SQLite board databases |
tokenization.py |
Converts frames strings to/from token sequences |
datasets.py |
PyTorch dataset adapters for training |
models.py |
Transformer encoder (grade predictor) and GPT (generator) |
generation.py |
Sampling, validity checks, frames reconstruction |
inference.py |
Model loading and inference helpers used by the webapp and demos |
evaluation.py |
Validity, novelty, and grade-consistency metrics |
visualization.py |
Board image overlays |
grades.py, metrics.py, utils.py |
Grade mapping, reporting helpers |
- Generated routes are machine-made candidates. Use at your own risk. They are not guaranteed to be safe, fun, or even physically possible.
- Grade predictions are estimates, not ground truth. Climbing grades are subjective, board-dependent, and noisy even in the training data.
- The hold sequence is a canonical ordering, not intended beta. The model sorts holds by role and position; this is not necessarily the order you would climb them.
- This is a research/hobby project, not affiliated with Tension Climbing or Kilter/Setter Closet.
This repo is the transformer/GPT follow-up to two earlier analysis projects:
Acknowledgement: the route generator draws on ideas from Andrej Karpathy's nanoGPT.
Board layouts, hold metadata, and route data are from the Tension Board 2 and Kilter Board apps, loaded via BoardLib. This project is unaffiliated with Tension Climbing or Kilter.
MIT — see LICENSE. Educational use.