Fix vLLM crash when max_gen_toks exceeds model context window - #153

Open

Ali-Elganzory wants to merge 3 commits into

mlfoundations:mainfrom

Ali-Elganzory:fix/vllm-context-window-crash

Commits on Jan 3, 2026

fix: cap max_gen_toks to prevent vLLM crash when exceeding context window
Ali-Elganzory
committed
a9a4fa0

Commits on Jan 4, 2026

Address PR feedback: add try-catch, prompt_length check, and constant
Ali-Elganzory
committed
977d755

Commits on Feb 6, 2026

fix: cap max_new_tokens to avoid degraded/undefined behavior when exceeding context window
Ali-Elganzory
committed
e93d565