Skip to content

feat(hub): add local-directory mode for offline model loading - #53

Merged
janpfeifer merged 4 commits into
gomlx:mainfrom
Niexiawei:feature/hub-local-dir
Jul 30, 2026
Merged

feat(hub): add local-directory mode for offline model loading#53
janpfeifer merged 4 commits into
gomlx:mainfrom
Niexiawei:feature/hub-local-dir

Conversation

@Niexiawei

Copy link
Copy Markdown
Contributor

Summary

  • Add hub.NewLocal(dir) / Repo.WithLocalDir(dir), letting a *hub.Repo read model files
    directly from a plain local directory (e.g. output of git clone or
    huggingface-cli download --local-dir) instead of downloading from HuggingFace Hub.
  • No network access is made in local-directory mode: DownloadInfo scans the directory to
    build the file listing, and DownloadFile(s) resolve repo-relative names to paths inside it.
    FileURL returns an explicit error since there is no remote URL in this mode.
  • Every existing consumer of *hub.Repomodels/transformer, models/safetensors,
    models/gguf, models/sam2, tokenizers/* — works unchanged, since they only depend on
    DownloadInfo/DownloadFile(s)/HasFile/IterFileNames/IterFileInfos, all of which are
    branched internally in hub.
  • Design rationale, alternatives considered, and known limitations (e.g. datasets is out of
    scope, standard-cache offline mode is a separate follow-up) are written up in
    docs/local-model-loading-plan.md.

Test plan

  • go build ./... and go vet ./... pass
  • go test ./hub/... passes, including 12 new tests in hub/local_test.go covering:
    forward-slash file names, .git/.cache exclusion, file sizes, HasFile,
    DownloadFile success/missing-file/path-traversal-safety, FileURL erroring in
    local mode, missing-directory error, and that scanning never mutates the local directory
  • go test ./... -short passes across the repo (one unrelated pre-existing failure in
    models/transformer on Windows, caused by an XLA PJRT auto-installer symlink-permission
    issue, not by this change)
  • Manually verified end-to-end against a real model (BAAI/bge-small-zh-v1.5) in a
    downstream consumer project: pointed it at a local snapshot directory via hub.NewLocal,
    ran embedding + similarity search with zero network calls, and got correct results

Niexiawei and others added 4 commits July 29, 2026 12:20
Add hub.NewLocal(dir) / Repo.WithLocalDir(dir) so a *hub.Repo can read
files straight from a plain local directory instead of downloading
from HuggingFace Hub, with no network access. DownloadInfo scans the
directory and DownloadFile(s) resolve paths within it; every existing
consumer of *hub.Repo works unchanged.
Removing artifact of the AI generation.
Removed design rationale reference from changelog entry.

@janpfeifer janpfeifer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @Niexiawei !

I removed the AI planning file, I hope you don't mind. Otherwise it looks great ... and I was meaning to do this for a while now, I'm so happy you already did it.

Merging here.

Comment thread docs/local-model-loading-plan.md Outdated
@janpfeifer
janpfeifer merged commit 792d867 into gomlx:main Jul 30, 2026
1 check passed
@janpfeifer

Copy link
Copy Markdown
Contributor

Btw, @Niexiawei , inspired by your PR, I went further in PR #54 and added the following:

  • Repo.Save(): save a repo to local disk.
  • Repo.DeleteCache(): erase a repo from cache (save space).
  • NewEmbed() load a repo that is embedded directly into the Go program (using //go:embed <dir>). This way one can deploy the model along with the go binary, in the same file.
  • The provided cmd/hubinfo CLI tool now also accepts:
    • -save: to save a repo to a local directory, so it can be used with NewLocal() (an alternative to huggingface-cli)
    • -delete_cache: free up the space used in cache for a repo.
    • -local: show info about a repo in a local directory.

I also updated the README.md with examples of using local directories.

Check it out!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants