Feature/add uqff support - #1
Conversation
# Conflicts: # src-tauri/src/chat/mod.rs
![sigit-code[bot]](https://avatars.githubusercontent.com/in/3492899?s=60&v=4){kind=small}
There was a problem hiding this comment.
Adds UQFF (pre-quantised) model support alongside the existing GGUF and ISQ paths. A new UQFF_MODELS table (Qwen 3 family) feeds chat_list_models and resolve_model_id, dispatched through a new ResolvedModel::Uqff arm to ChatEngine::load_uqff_model. Download progress is now observable: blobs_dir_size gains an include_incomplete flag distinguishing completeness (ignore .part/.lock) from progress (count them), exposed via a new chat_download_progress command and polled from Settings to show a real percentage bar. CI clones a sibling onde checkout because Cargo.toml patches onde to a local path for the unreleased load_uqff_model API.
Start with mod.rs: the blobs_dir_size semantics split and the ResolvedModel dispatch are the core. Note the CI/Cargo.toml patch is explicitly temporary and must be reverted once onde publishes.
Automated review by siGit Code · commit e64c41a
| description: sanitize_description(entry.description), | ||
| approx_memory: entry.approx_memory.to_string(), | ||
| size_bytes: Some(entry.expected_size_bytes), | ||
| is_downloaded: is_model_downloaded(entry.id, entry.expected_size_bytes), |
There was a problem hiding this comment.
warning — Two 4B variants (Qwen3-4B-Instruct-2507 and Qwen3-4B-Thinking-2507) share the identical expected_size_bytes 3_040_959_629 as the base 4B; if that value was copied rather than measured per-repo, is_downloaded/progress will be off for those two.
| let entry = UQFF_MODELS.iter().find(|m| m.id == id)?; | ||
| Some(UqffModelConfig { | ||
| model_id: entry.id.to_string(), | ||
| files: vec!["q4k-0.uqff".to_string()], |
There was a problem hiding this comment.
warning — files is hardcoded to q4k-0.uqff for every entry; if any repo shards the q4k weights across more than one file (e.g. q4k-1.uqff), the load and the size accounting silently miss the rest.
| } | ||
| let active = true; | ||
| const tick = async () => { | ||
| const bytes = await getDownloadProgress(selectedId).catch(() => null); |
There was a problem hiding this comment.
nit — If getDownloadProgress returns 0 during a real in-flight download (unknown id / empty cache also return 0), the bar shows 0% indistinguishably from a stalled transfer, matching the doc but worth confirming the id passed matches the loading model's cache id.
| - name: Check out the Onde engine as a sibling checkout | ||
| run: | | ||
| git clone --depth 1 \ | ||
| --branch feature/add-UQFF-support \ |
There was a problem hiding this comment.
nit — The clone hardcodes branch feature/add-UQFF-support; once that branch merges/deletes on the onde side, CI on this branch breaks until the patch block is removed as noted.
There was a problem hiding this comment.
Adds UQFF (pre-quantised Qwen 3) model support: a new UQFF_MODELS table with per-model metadata, a ResolvedModel::Uqff variant and load path via onde's load_uqff_model, plus a download-progress command (chat_download_progress) that reads bytes-on-disk (including .part files) from the HF cache. The frontend polls this to turn the loading spinner into a real percentage. blobs_dir_size gains an include_incomplete flag so completeness checks exclude .part/.lock while progress includes them, with tests covering the threshold regression. CI temporarily clones the onde sibling checkout because Cargo.toml now patches onde to a local path — reviewers should note this and the patch must be reverted once onde ships UQFF.
Automated review by siGit Code Review · commit 15eef14
| { | ||
| use super::UQFF_MODELS; | ||
| for entry in UQFF_MODELS { | ||
| if entry.desktop_only && !cfg!(any(target_os = "macos", target_os = "windows")) { |
There was a problem hiding this comment.
warning — desktop_only filters to macOS/Windows only, but the model table is compiled for iOS/Android/Windows too; non-desktop-only UQFF models are still offered on mobile, so verify the huge non-desktop_only entries (e.g. 32B ~19.5 GB) are actually intended to appear on phones.
| target_os = "windows" | ||
| ))] | ||
| pub(crate) fn uqff_config_for_model_id(id: &str) -> Option<UqffModelConfig> { | ||
| let entry = UQFF_MODELS.iter().find(|m| m.id == id)?; |
There was a problem hiding this comment.
warning — Every 30B-A3B / MoE / Coder entry shares the identical expected_size_bytes (17_640_189_769) and several 4B variants share 3_040_959_629; if these are copy-paste placeholders rather than true measured sizes, is_model_downloaded and the progress percentage will be wrong for those models.
| target_os = "windows" | ||
| )))] | ||
| #[tauri::command] | ||
| pub async fn chat_download_progress(_model_id: String) -> u64 { |
There was a problem hiding this comment.
nit — The cfg(not(...)) fallback lists the same four targets; simpler to just use the always-defined model_downloaded_bytes if it were gated identically, but as-is the duplicated cfg lists must stay perfectly in sync across three files.
| } | ||
| let active = true; | ||
| const tick = async () => { | ||
| const bytes = await getDownloadProgress(selectedId).catch(() => null); |
There was a problem hiding this comment.
nit — The 1s poll continues across model switches: selectedId change re-triggers the effect, but a slow in-flight getDownloadProgress for the previous id could still resolve; active guards state but the awaited call for the old model may briefly overlap. Minor given the guard.
1 participant
What & why
Add support for UQFF model format.
Blocked by ondeinference/onde#10
How it was tested
Checklist
make lintpasses (clippy + tsc, no warnings)make fmtappliedmake buildcompiles