docs: explain how the model families diverge, in one place - #30
Open
Rebreda wants to merge 1 commit into
Open
Conversation
Three families are supported now and the differences between them were spread across a paragraph in the AMD guide, a docstring, and the code. Someone adding a fourth had no way to know what the complete list of differences was. New docs/architectures.md states it once: what is supported, the four fields that actually differ, and why each exists. Everything in it is a real field on Architecture, so the page maps to the code rather than describing it. It also covers the two divergences that are handled by falling back rather than by a field, since they are invisible otherwise: offline Moonshine's tokenizer exposes no special tokens so the pad token is read from the model config, and Whisper names its output projection out_proj while both Moonshine variants use o_proj. The base model section of finetune-amd.md now says which families exist and links here, rather than repeating a two-family comparison that was already out of date. Four tests keep the page honest. A stale table here is worse than no table, because it tells someone they have covered every difference when they have not, so one test fails if a field is added to Architecture without a matching row.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three families are supported now, and the differences between them were spread across a paragraph in the AMD guide, a docstring, and the code. Someone adding a fourth had no way to know what the complete list of differences was.
New
docs/architectures.mdstates it once.The table
Every row is a real field on
Architecture, so the page maps to the code rather than describing it:whispermoonshinemoonshine_streamingfeature_keyinput_featuresinput_valuesinput_valuespad_featuresFalseTrueTruepad_to_multiple80supports_language_and_taskTrueFalseFalseThe invisible divergences
Two differences are handled by falling back rather than by a field, so they do not appear in the table and would otherwise be undiscoverable:
Offline Moonshine's tokenizer exposes no special tokens at all, so the pad token is read back from the model config. Whisper and streaming Moonshine both have one.
Whisper names its output projection
out_projwhile both Moonshine variants useo_proj. The default LoRA targets avoid it, so this only matters if you setlora_target_modulesyourself.Also stated plainly, because it caught me out: Lemonade serves
moonshine-ai/moonshine-streaming, which is ONNX-only and cannot be fine-tuned. Serving and fine-tuning are different artifacts.Keeping it honest
Four tests. A stale table here is worse than no table, because it tells someone adding a family that they have covered every difference when they have not. One test enumerates
dataclasses.fields(Architecture)and fails if a field is added without a matching row.Elsewhere
finetune-amd.md's base model section now names the three families and links here, instead of repeating a two-family comparison that was already out of date. README gains the page in its docs table.386 passed and 14 skipped on 3.11 and on a dev-only install, 406 with the extras. All doc links resolve.