Skip to content

Add adept.cloud.submit: shared AWS Batch submit client - #321

Open
joglekara wants to merge 1 commit into
mainfrom
feat/cloud-submit
Open

Add adept.cloud.submit: shared AWS Batch submit client#321
joglekara wants to merge 1 commit into
mainfrom
feat/cloud-submit

Conversation

@joglekara

Copy link
Copy Markdown
Member

Closes #316. Design and rationale: ergodicio/continuum-infra#47. Runtime half: ergodicio/continuum-infra#61.

The Batch sim images (sim-cpu, sim-gpu) carry the environment only — application code arrives at run time as a content-addressed S3 tarball, so a one-line physics edit costs an upload instead of a docker build, an ECR push and a cdk deploy. sim-runner/submit.py in the infra repo was the proof-of-concept form of the submit half; this is it as a library, in the repo every sim repo already depends on.

Surface

from adept.cloud import submit

result = submit(cmd="python run.py --cfg configs/foo", queue="gpu", array=16, extras="gpu")
print(result.job_id, result.adept_sha)

Also python -m adept.cloud --cmd ... --queue ... [--dry-run], and array_index() for the worker side (returns 0 outside an array job, so one entry point runs both on Batch and by hand).

What carried over from the proof of concept

The four requirements #316 recorded, each found by testing rather than design:

  • uv.lock and pyproject.toml ship regardless of git status. kinetic-srs gitignores its lock, so a bundler that merely respects .gitignore ships no lock and the runner has nothing to install from.
  • Reproducible archives. tarfile's w:gz stamps the gzip header with the current time and offers no override, so identical trees hashed differently and content addressing never hit. Built uncompressed, gzipped with mtime=0, member metadata normalised.
  • Committed outputs dropped by suffix, loudly. These repos commit their outputs; the count and size skipped are printed on every submit, and include_suffixes puts one back when a run needs it as input.
  • Array support, so an N-point scan is one submission.

Beyond it

  • extra_files={arcname: bytes|str} ships content generated at submit time — a scan manifest, a resolved config — that is not in the tree at all. It participates in the digest, so two submissions of the same code with different manifests are different bundles. This is what ml-for-lpi's array path is built on (Run scans on AWS Batch array jobs via the generic sim runner ml-for-lpi#11).
  • SubmitResult returns the job id, code_uri, digest, array size and adept_sha, resolved from the lock inside the bundle. Both sim repos declare adept @ git+…@main, i.e. unpinned, so today a run's adept version is whatever was current when its environment was last built, with no record of which. This makes it recordable per run.
  • job_definition defaults per queue (gpu/gpu-48gsim-gpu, cpusim-cpu, cpu-hmemsim-cpu-hmem), matching what continuum-infra#62 deployed.
  • array=1 degrades to a plain job — Batch has no array of size 1.
  • repo_prefix() keys bundles on the main worktree's name, so iterating in a linked git worktree does not scatter one repo's objects across prefixes (the lifecycle rule and any manual audit both read by prefix).

Testing

18 tests in tests/test_cloud/test_bundle.py, no AWS needed — determinism, the gitignored lock, suffix drops, extra_files, worktree prefixes, and the submit request itself against fake S3/Batch clients (array size, parameters, no re-upload of an identical bundle, dry run touching nothing). ruff check and ruff format clean. Docs: docs/source/usage/cloud.md.

Nothing in adept/__init__.py imports cloud, so this adds no import cost to a solver run.

🤖 Generated with Claude Code

The Batch sim images carry the environment only; application code arrives at run time as a
content-addressed S3 tarball, so a physics edit costs an upload rather than a docker build, an
ECR push and a cdk deploy. This is the submit half of that contract (runtime half:
continuum-infra/sim-runner/bootstrap.sh).

It lives in adept because every sim repo that submits already depends on adept, and adept already
depends on boto3 -- so the bundling rules live in one place instead of being vendored per repo.

Carries over the four requirements the infra proof of concept established by testing:
force-include uv.lock/pyproject.toml (some repos gitignore the lock, and the runner installs from
it), reproducible archives (tarfile's w:gz stamps the gzip header with the current time, so
content addressing never hit), loud suffix-drop of committed outputs, and array support.

Beyond the proof of concept:
- extra_files ships content generated at submit time (a scan manifest, a resolved config), which
  participates in the digest
- SubmitResult returns the job id, code_uri, digest, array size, and the adept sha resolved from
  the lock in the bundle -- the only record of which physics ran, since the sim repos pin @main
- job_definition defaults per queue; array=1 degrades to a plain job, which Batch requires
- repo_prefix keys bundles on the main worktree's name, so iterating in a linked worktree does not
  scatter one repo's objects across prefixes

Closes #316

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add adept.cloud.submit: shared AWS Batch submit client for the sim repos

1 participant