From 777093af0489d98876b406725dd553f5d1cdf5b6 Mon Sep 17 00:00:00 2001
From: Maxim Usov <m.usov@enbisys.com>
Date: Thu, 4 Jun 2026 17:04:21 +0700
Subject: [PATCH] Fix README: correct token IDs for LLaDA tokenizer
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The README examples used token IDs from the Llama 3 tokenizer
(vocab_size=128256), but LLaDA uses its own tokenizer with
vocab_size=126080. Token 128000 is out of bounds for the model's
embedding matrix (126464 rows), causing an assertion failure in
ggml_compute_forward_get_rows and a crash.

Changes:
- Bash example: 128000,... → 2372,341,268,7706,300,11406,30
- Python snippet: full model path + trust_remote_code=True
  (huggingface-cli can't resolve 'LLaDA-8B-Instruct' without
  the GSAI-ML namespace)
- C++ API example: correct token IDs matching the example prompt
- Added warning about using the correct tokenizer per model

Verified: the corrected tokens produce 'The capital of France is Paris.'
---
 README.md | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/README.md b/README.md
index 2da63eb..e6a460a 100644
--- a/README.md
+++ b/README.md
@@ -108,11 +108,14 @@ cmake --build build -j$(nproc)
 ```bash
 # Pre-tokenize your prompt (diffuse-cpp does not include a tokenizer)
 # Use transformers or tiktoken to get token IDs as a comma-separated list
+# CRITICAL: Use the correct tokenizer for your model.
+#   LLaDA → GSAI-ML/LLaDA-8B-Instruct
+#   Dream → Dream-org/Dream-v0-Instruct-7B
 
-# Generate text (example with token IDs)
+# Generate text (example for LLaDA with "What is the capital of France?")
 ./build/diffuse-cli \
     -m model.gguf \
-    --tokens "128000,3923,374,279,6864,315,9822,30" \
+    --tokens "2372,341,268,7706,300,11406,30" \
     -n 256 \
     -s 16 \
     -t 12 \
@@ -123,7 +126,7 @@ cmake --build build -j$(nproc)
 
 ```python
 from transformers import AutoTokenizer
-tokenizer = AutoTokenizer.from_pretrained("LLaDA-8B-Instruct")
+tokenizer = AutoTokenizer.from_pretrained("GSAI-ML/LLaDA-8B-Instruct", trust_remote_code=True)
 tokens = tokenizer.encode("What is the capital of France?")
 print(",".join(map(str, tokens)))
 ```
@@ -272,7 +275,7 @@ params.remasking = diffuse_remasking::ENTROPY_EXIT;
 params.entropy_threshold = 1.5f;
 
 // Generate
-std::vector<int32_t> prompt_tokens = {128000, 3923, 374};
+std::vector<int32_t> prompt_tokens = {2372, 341, 268, 7706, 300, 11406, 30};
 auto output = diffuse_generate(ctx, prompt_tokens, 256, params);
 
 // Cleanup