Send the datasource when downloading a blob

[guysmoilov]

The server now stores data engine metadata blobs under a per-datasource prefix instead of a repo-wide one, so that deleting a datasource can actually delete its blobs.

The server can still serve a blob when it isn't told which datasource it belongs to — but only by checking the legacy location and then searching the repo's datasources one at a time. We already know the datasource here, so we may as well say so.

The change

blob_path appends ?datasource={id}, which lets the server go straight to the blob.

This is an optimization, not a requirement. Nothing breaks without it, in either direction:

• Old client + new server: no parameter, so the server searches. Correct, just does a bit more work.
• New client + old server: an unknown query parameter, which the old server ignores.
• Datasource with no id yet (never saved): the parameter is omitted rather than sent as datasource=None.

So this can ship on its own schedule; the server PR does not depend on it.

Testing

Two new tests in tests/data_engine/test_datasource_state.py: the url carries the datasource, and a datasource without an id still produces a usable url. Full suite passes (355 tests).

Also fixed the annotation test's blob mock, which derived the blob hash from the last path segment of the download url and so swallowed the query string. It now parses the url the way the server does — path for the hash, query separately. The real client is unaffected: it caches by hash, independently of the url.

[coderabbitai]

📝 Walkthrough

Walkthrough

DatasourceState.blob_path now adds a datasource query parameter when an ID exists, preserves the base URL otherwise, and updates related tests to handle query-bearing blob URLs.

Changes

Datasource-aware blob URLs

Layer / File(s)	Summary
Blob URL generation and validation dagshub/data_engine/model/datasource_state.py, tests/data_engine/test_datasource_state.py, tests/data_engine/annotation_import/test_annotation_parsing.py	blob_path conditionally appends ?datasource=<id>, with tests covering identified and unsaved datasource states. Blob hash extraction now ignores URL query parameters.

Estimated code review effort: 2 (Simple) | ~10 minutes

Poem

I’m a rabbit hopping through the URL night,
Adding datasource tags just right.
Hashes stay clean when queries appear,
Tests guard the path with a twitchy ear.
Hop, hop—no None is near!

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly states the main change: including the datasource when downloading a blob.
Description check	✅ Passed	The description matches the changeset and explains the blob URL update, compatibility, and tests.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feature/datasource-scoped-blob-path

---

_{Comment @coderabbitai help to get the list of available commands.}