Ting Mahjong Agent

Ting is a Chinese Standard Mahjong bot that makes decisions through a neural network. The core bot is split into five key files:

• bot.py is the entry point for the bot. It handles the raw input, updates the game state, and returns a decision.
• state.py parses the input and stores all relevant information in the GameState class.
• features.py extracts the features from the game state.
• policy.py handles the decision-making by retrieving the features and passing it into the neural model.
• model.py defines the neural network, allowing for training and inference.

Features Schema

The features are split into tile planes and a meta vector, as defined in features.py.

• Tile planes (21 x 34):
  • 4 hand-count threshold planes (>=1, >=2, >=3, >=4).
  • 4 meld planes.
  • 4 discard planes.
  • 4 unseen-count threshold planes (>=1, >=2, >=3, >=4).
  • 1 one-hot plane for the most recently played or drawn tile.
  • 1 plane marking tiles that improve hand (decrease shanten).
  • 3 oracle planes that estimate opponent hands.
• Meta vector (31 values):
  • Seat and prevalent-wind one-hots.
  • Decision phase (draw / discard-response / bugang-response).
  • Relative last actor, last request action.
  • Normalized scalars for flowers, meld count, game progress, shanten, tenpai flag, acceptance count, and the can-win / win-fan signal.

Model Architecture

model.py defines a residual policy-value network:

• The tile planes are reshaped onto a 4x9 grid (suit rows + honor row).
• A Conv2D steam layer extracts the low-level features.
• Several residual blocks are used to refine the features.
• They are then flattened and combined with the meta vector.
• A hidden representation is produced for final decision-making.

The model predicts various things:

• Action-family logits (PASS/HU/GANG/PLAY/BUGANG/PENG/CHI) that represent which action to use.
• arg1_all and arg2_all that represent which tiles to use for the action.
• value that represents the estimated value of the position.
• win_logit that represents the likelihood of the player winning.

The model makes s decision by scoring the family + arg1 + arg2 logits. The policy uses softmax over the legal actions only to choose the highest scoring action.

Training

1. Training data

The data is stored as data/data.txt (98,209 real rounds; data/sample.txt is a 16-round preview).

botzone_ingest.py replays every round, reconstructs each player's information state, and emits trajectory JSONL. It includes the features, claim decisions, ignored HU/GANG declarations, per-player final-score rewards, and steps_from_end for credit decay.

python src/botzone_ingest.py --input data/data.txt --output data/botzone.jsonl --verbose

2. Supervised training

imitation.py trains the specified dataset(s).

python src/imitation.py train-cnn \
    --dataset data/botzone.jsonl:1.0 \
    --out src/model.h5 --epochs 16 --channels 128 --blocks 12 --hidden-size 512 \
    --batch-size 1024 --learning-rate 0.0005 --device auto --verbose

eval-cnn reports top-k masked accuracy, masked CE, value MSE, win-head accuracy, and ECE calibration, split by decision vs forced states:

python src/imitation.py eval-cnn --dataset data/botzone.jsonl --model src/model.h5

3. Reinforcement learning and self-play

rl_self_play.py fine-tunes a model using a PPO policy. The model can play against the rule-based baseline, 27 pre-trained models in data/models/, or mirror copies of the current candidate.

python src/rl_self_play.py ppo-train --model src/model.h5 --games 512 \
    --eval-games 64 --device auto \
    --finalist-dir data/models --finalist-prob 0.5 \
    --league-dir checkpoints/league --self-play-prob 0.2

finalist-eval lets the model play against the pre-trained models, which is the evaluation metric:

python src/rl_self_play.py finalist-eval --model src/model.h5 \
    --finalist-dir data/models --games 256 --verbose

The model can get promoted (see model_governance.py) and get snapshotted into --league-dir so later runs face them as opponents.

Local Testing

Run a single game and print the final state:

python src/local_game.py --games 1 --seed 42 --tui --model-path src/model.h5

Pipe a JSON payload through the bot:

echo '{"requests": ["0 0 0", "1 0 0 0 0 W1 W2 W3 B4 B5 B6 T7 T8 T9 J1 J1 F1 F2", "2 W4"], "responses": ["PASS", "PASS"]}' | python src/bot.py

Run the full test suite:

python -m unittest -q