Add adept.cloud.submit: shared AWS Batch submit client - #321
Open
joglekara wants to merge 1 commit into
Open
Conversation
The Batch sim images carry the environment only; application code arrives at run time as a content-addressed S3 tarball, so a physics edit costs an upload rather than a docker build, an ECR push and a cdk deploy. This is the submit half of that contract (runtime half: continuum-infra/sim-runner/bootstrap.sh). It lives in adept because every sim repo that submits already depends on adept, and adept already depends on boto3 -- so the bundling rules live in one place instead of being vendored per repo. Carries over the four requirements the infra proof of concept established by testing: force-include uv.lock/pyproject.toml (some repos gitignore the lock, and the runner installs from it), reproducible archives (tarfile's w:gz stamps the gzip header with the current time, so content addressing never hit), loud suffix-drop of committed outputs, and array support. Beyond the proof of concept: - extra_files ships content generated at submit time (a scan manifest, a resolved config), which participates in the digest - SubmitResult returns the job id, code_uri, digest, array size, and the adept sha resolved from the lock in the bundle -- the only record of which physics ran, since the sim repos pin @main - job_definition defaults per queue; array=1 degrades to a plain job, which Batch requires - repo_prefix keys bundles on the main worktree's name, so iterating in a linked worktree does not scatter one repo's objects across prefixes Closes #316 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #316. Design and rationale: ergodicio/continuum-infra#47. Runtime half: ergodicio/continuum-infra#61.
The Batch sim images (
sim-cpu,sim-gpu) carry the environment only — application code arrives at run time as a content-addressed S3 tarball, so a one-line physics edit costs an upload instead of adocker build, an ECR push and acdk deploy.sim-runner/submit.pyin the infra repo was the proof-of-concept form of the submit half; this is it as a library, in the repo every sim repo already depends on.Surface
Also
python -m adept.cloud --cmd ... --queue ... [--dry-run], andarray_index()for the worker side (returns 0 outside an array job, so one entry point runs both on Batch and by hand).What carried over from the proof of concept
The four requirements #316 recorded, each found by testing rather than design:
uv.lockandpyproject.tomlship regardless of git status. kinetic-srs gitignores its lock, so a bundler that merely respects.gitignoreships no lock and the runner has nothing to install from.tarfile'sw:gzstamps the gzip header with the current time and offers no override, so identical trees hashed differently and content addressing never hit. Built uncompressed, gzipped withmtime=0, member metadata normalised.include_suffixesputs one back when a run needs it as input.Beyond it
extra_files={arcname: bytes|str}ships content generated at submit time — a scan manifest, a resolved config — that is not in the tree at all. It participates in the digest, so two submissions of the same code with different manifests are different bundles. This is what ml-for-lpi's array path is built on (Run scans on AWS Batch array jobs via the generic sim runner ml-for-lpi#11).SubmitResultreturns the job id,code_uri, digest, array size andadept_sha, resolved from the lock inside the bundle. Both sim repos declareadept @ git+…@main, i.e. unpinned, so today a run's adept version is whatever was current when its environment was last built, with no record of which. This makes it recordable per run.job_definitiondefaults per queue (gpu/gpu-48g→sim-gpu,cpu→sim-cpu,cpu-hmem→sim-cpu-hmem), matching what continuum-infra#62 deployed.array=1degrades to a plain job — Batch has no array of size 1.repo_prefix()keys bundles on the main worktree's name, so iterating in a linkedgit worktreedoes not scatter one repo's objects across prefixes (the lifecycle rule and any manual audit both read by prefix).Testing
18 tests in
tests/test_cloud/test_bundle.py, no AWS needed — determinism, the gitignored lock, suffix drops,extra_files, worktree prefixes, and the submit request itself against fake S3/Batch clients (array size, parameters, no re-upload of an identical bundle, dry run touching nothing).ruff checkandruff formatclean. Docs:docs/source/usage/cloud.md.Nothing in
adept/__init__.pyimportscloud, so this adds no import cost to a solver run.🤖 Generated with Claude Code