Skip to content

Add ForcedSequenceLogitProcessor (teacher-forcing custom logit processor)#32

Open
DavidBellamy wants to merge 1 commit into
prodfrom
forced-sequence-logit-processor
Open

Add ForcedSequenceLogitProcessor (teacher-forcing custom logit processor)#32
DavidBellamy wants to merge 1 commit into
prodfrom
forced-sequence-logit-processor

Conversation

@DavidBellamy

Copy link
Copy Markdown
Collaborator

What

Adds a general-purpose ForcedSequenceLogitProcessor to python/sglang/srt/sampling/custom_logit_processor.py. Per request, pass custom_params={"forced_output_ids": [int, ...]}; at decode step t it forces the next token to forced_output_ids[t] by masking all other logits, then samples normally past the end of the list.

Why

Lets two server configurations run a byte-identical decode workload so the only variable is the configuration under test (a caching flag, a kernel, a quantization mode), and supports reproducible / replayed decoding. It's a natural sibling of the existing ThinkingBudgetLogitProcessor, DisallowedTokensLogitsProcessor, and DeepseekOCRNoRepeatNGramLogitProcessor in the same file.

Usage

  • Launch the server with --enable-custom-logit-processor.
  • Per request (native /generate or /v1/chat/completions):
    • custom_logit_processor = ForcedSequenceLogitProcessor.to_str()
    • sampling_params.custom_params = {"forced_output_ids": [...]}
    • max_new_tokens = len(forced_output_ids)

Standalone; no dependency on other in-flight changes.

@DavidBellamy DavidBellamy requested a review from a team June 5, 2026 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant