fix benchmark template

[Dogacel]

Motivation

Chat template of some models (gpt-oss, qwen3...) are not rendered correctly in SGLang, causing
accuracy to be reported incorrectly.

This patch checks if SGLang consists a template for the given model, if not it handles chat formatting manually using HF tokenizer and sends the templated text directly to SGLang.

Modifications

1. Auto detect chat template format and override it if sglang fails to auto-detect.
2. Fix GSM8K benchmark not applying the chat format correctly.

Also adds 3 new benchmarks:

• SVAMP: Simple Variations on Arithmetic Math word Problems - https://github.com/arkilpatel/SVAMP
• GSM1K: Basically GSM8K but designed to measure overfit on GSM8K.
• Alpaca: Tatsu-lab's conversational fine-tuning dataset - https://huggingface.co/datasets/tatsu-lab/alpaca.

Related Issues

Fixes #551

Accuracy Test

Tested on 3 model families with different template formats gpt-oss-20b, qwen3 8B, llama3.1 8B.

Any model that reported their accuracy using the previous benchmark should ensure SGLang was able to auto-detect the template. Moreover all models should re-report their GSM8K benchmark results because not applying template correctly causes accuracy drops.

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://sgl-fru7574.slack.com/archives/C09784E3EN6 to discuss your PR.

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!