Skip to content

Add pos3 CLI (ls, download, upload) and bump to 0.3.1#11

Merged
vertix merged 16 commits into
mainfrom
claude/add-documentation-trVyZ
May 30, 2026
Merged

Add pos3 CLI (ls, download, upload) and bump to 0.3.1#11
vertix merged 16 commits into
mainfrom
claude/add-documentation-trVyZ

Conversation

@vertix

@vertix vertix commented May 23, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add a pos3 console-script CLI with three subcommands: ls, download, upload. Uses the existing profile registry so URLs and auth resolve the same way they do for the Python API.
  • Bump version 0.3.00.3.1 and add a [0.3.1] CHANGELOG entry covering the CLI.
  • No changes to existing public Python API or to requires-python (stays >=3.11).

Test plan

  • pip install -e . and confirm pos3 --help lists ls, download, upload
  • pos3 ls <path> against a configured profile prints expected entries
  • pos3 download <remote> <local> writes the file and exits 0
  • pos3 upload <local> <remote> uploads and is visible via ls
  • Run the existing test suite — no regressions

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas


Generated by Claude Code

claude added 2 commits May 21, 2026 15:32
Introduces a console-script entry point so common pos3 operations are
one-liners from the shell instead of requiring a Python script.

- pos3 ls <prefix> [-r] [--profile NAME] — one full s3:// URL per line on stdout.
- pos3 download <url> [--local PATH] [--delete] [--exclude PATTERN]... [--profile NAME]
  — prints only the resulting local path to stdout (progress + logs go to
  stderr), so data_dir=$(pos3 download s3://bucket/dataset/) is safe.
- pos3 upload <url> [--local PATH] [--delete] [--exclude PATTERN]... [--profile NAME]
  — one-shot upload (no background loop, no interval). Source defaults to the
  cache path pos3 download would have produced; errors if the source is missing.

--delete defaults OFF in the CLI even though the Python API defaults to True:
CLI defaults are conservative for interactive shell use. --profile is honored
alongside the URL form s3://<profile>@bucket/...; URL wins on conflict, matching
existing Python precedence.

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas
v0.3.0 is already released; the pos3 CLI added in the previous commit
ships as 0.3.1. CHANGELOG section moves from [Unreleased] to [0.3.1].

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9da3a7b6be

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pos3/cli.py
Comment thread pos3/cli.py
The flag is accepted only on the two transfer subcommands (ls rejects it).
Dry-run reuses the existing _compute_sync_diff so the plan reflects what a
real run would do; output is `aws s3 sync --dryrun`-style one-line-per-file
on stdout. No transfers, deletes, or directory creation happen. Synthesized
directory entries from _scan_s3 are skipped so the output is file-level.

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: db93dd27a0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pos3/cli.py
The Python API treats non-S3 inputs to download()/upload() as a local-path
passthrough — useful when calling code is polymorphic over local/remote
inputs, but in the CLI it meant `pos3 download bucket/path` (no `s3://`)
exited 0 and printed a local path without transferring anything. The CLI
help text on both commands already says "Source/Destination S3 URL", so a
typo could silently break shell pipelines.

Add an explicit s3:// guard in both _cmd_download and _cmd_upload. Tighten
the dry-run wording in the cli docstring, README, and CHANGELOG: dry-run
performs no transfers and no deletes, but the cache root is still mkdir'd
on mirror() entry the same way it is for any pos3 invocation.

`ls` is unchanged — it documents and supports both s3:// and local paths.

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a60803fb31

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pos3/cli.py
Comment thread pos3/cli.py
Per-worker failures in _process_futures were logged and swallowed, so a
download() / upload() with a failed S3 GET or PUT (403, network blip,
etc.) returned normally. Library callers got a path to a partial cache;
the new pos3 CLI exited 0 after printing that path, breaking the
`data_dir=$(pos3 download ...)` contract codex flagged.

Introduce pos3.TransferError(operation, failures). _process_futures now
collects per-worker exceptions and raises one TransferError once all
futures have been drained (we keep the "do as much as we can, then
report" semantics rather than fail-fast cancelling pending work). The
existing Mirror.download error handler already re-raises arbitrary
exceptions; for upload the failure surfaces from the mirror() context's
final sync. The CLI main() catches TransferError alongside ValueError
and exits 1 with the failure on stderr.

Tests: two new TestCliTransferFailures cases inject side_effect on
download_file / upload_file and assert rc == 1, no path on stdout for
download.

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e9da8ae8c9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pos3/__init__.py
The previous commit made _process_futures raise, which was the right call
for one-shot download/upload (CLI must exit non-zero, library callers
must not silently get partial caches). But it also fired inside the
_background_worker daemon's _sync_uploads(due) call — a single transient
upload_file failure under `upload(..., interval=N)` would now kill the
thread and stop all future interval syncs for the rest of the mirror
context.

Wrap the _sync_uploads call inside the worker loop in try/except: log
and continue. Best-effort retry-next-tick is the right model for
periodic syncs. _final_sync (context exit) and one-shot download() still
propagate, so CLI behavior is unchanged.

Test: test_background_worker_survives_transfer_error registers an upload
with interval=1, makes the first upload_file call raise, and sleeps 2.5s.
call_count >= 2 confirms the daemon survived and retried.

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b99561e5ed

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pos3/__init__.py
Three connected cleanups motivated by the codex review cycle:

1. Add pos3.TransferPlan (frozen dataclass with to_copy / to_delete) and
   Mirror.plan_download / Mirror.plan_upload. These compute the set of
   (source, destination) copies and target deletes a real call would
   perform, without performing any of them. The CLI's _print_*_plan
   helpers shrink from ~25 lines reaching into 6 private helpers to ~12
   lines calling one public method. Library callers can now ask "would
   download() do anything?" the same way the CLI does.

2. Drop the eager cache_root.mkdir from _Mirror.__init__. The leaf
   directory is still mkdir'd on demand by _put_locally (it does
   target.parent.mkdir(parents=True)), so download() / upload() behavior
   is unchanged. Dry-run and plan_* paths are now genuinely side-effect
   free — closes codex's earlier P2 for real, not just in docs.

3. _final_sync(had_error=True) now catches TransferError from the
   cleanup sync, logs it, and lets the original app exception propagate.
   Without this, an experiment that raises AppError followed by a failed
   cleanup upload would see TransferError as the top-level cause —
   regression from the pre-TransferError logged-and-continued behavior.
   Codex's most recent P2 ("Preserve the original exception during
   sync_on_error cleanup").

Also documents _process_futures' asymmetric error model so the
background-worker try/except is no longer surprising.

Tests added: TestPlan (plan_download / plan_upload + non-S3 rejection),
TestMirrorConstructorIsSideEffectFree, and
TestFinalSyncPreservesOriginalException.

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 95f493adb1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pos3/__init__.py Outdated
A trailing slash on the input URL — e.g. pos3 download -n s3://bucket/data/
— caused plan output to emit s3://bucket/data//file.txt because
_parse_s3_url("s3://bucket/data/") returns prefix "data/" and
_make_s3_key("data/", info) then appends another slash. Mirror.download
already normalizes via _normalize_s3_url before parsing, so the dry-run
plan was misstating exact keys vs what a real run would transfer.

Fix: in plan_download and plan_upload, call _parse_s3_url on
_normalize_s3_url(remote) instead of on the raw URL. Single new line
per method.

Tests: test_plan_download_normalizes_trailing_slash_in_url asserts
sources == ["s3://bucket/data/file.txt"]; the upload variant also
covers the to_delete branch since it goes through _make_s3_key too.

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

pos3/pos3/__init__.py

Lines 606 to 607 in 8479ea5

if key:
key = key + "/"

P2 Badge Let ls probe exact S3 object keys

When ls is called on an S3 URL that names a single object, e.g. after uploading a local file to s3://bucket/results.json, this unconditional slash changes the lookup to results.json/ and _list_s3_objects() skips its existing head_object exact-key probe because the key now ends with /. The command therefore lists nothing even though the object exists; _list_s3_objects() already avoids spurious prefix matches by probing the raw key first and only adding / after a 404, so ls should not force the slash here.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ls() unconditionally appended "/" to a non-empty key before calling
_scan_s3 / _list_s3_objects. But _list_s3_objects already has the right
"try as exact object first, fall back to directory" logic — it does a
head_object on the raw key and only adds "/" after a 404. Forcing "/"
up front bypassed that probe, so `pos3 ls s3://bucket/results.json`
returned nothing for an existing object (head_object skipped, list with
prefix "results.json/" finds nothing).

Drop the forced slash. Add a single-object branch to the ls loop:
_scan_s3 yields FileInfo(relative_path="", is_dir=False) for an exact
key match, which we now emit as the input URL. Directory-listing
behavior is unchanged: when the key is a real prefix, head_object 404s,
_list_s3_objects appends "/" and lists, and we hit the existing
relative_path-based reconstruction.

The spurious-prefix concern from the old comment ("droid/recovery"
matching "droid/recovery_towels") is already covered by
_list_s3_objects' post-404 append.

Test: test_ls_single_object asserts `pos3 ls s3://bucket/results.json`
prints exactly that URL on stdout when head_object returns 200.

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas

vertix commented May 28, 2026

Copy link
Copy Markdown
Contributor Author

Re: Codex's P2 on pos3/__init__.py:606-607 ("Let ls probe exact S3 object keys") — confirmed and fixed in 5334ec1.

ls() unconditionally appended / to a non-empty key, but _list_s3_objects at L823 already does the right thing: it head_objects the raw key first and only falls back to a prefix + "/" listing on 404. Forcing the slash up front suppressed the exact-key probe, so pos3 ls s3://bucket/results.json returned nothing for an existing object.

Two changes:

  1. Drop the forced slash in _Mirror.ls() — let _list_s3_objects make the call.
  2. Add a single-object branch to the ls loop: _scan_s3 yields FileInfo(relative_path="", is_dir=False) when the key is an exact object match; we now emit the input URL for that case. The root-directory marker (relative_path=="" and is_dir=True) is still filtered out.

The original "spurious prefix" concern in the deleted comment (droid/recovery matching droid/recovery_towels) is preserved because _list_s3_objects' post-404 path still appends / before listing.

New test test_ls_single_object: mocks head_object to return 200, asserts pos3 ls s3://bucket/results.json prints exactly that URL. Existing directory-listing tests untouched. Full suite: 123 passed.


Generated by Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5334ec1f4e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pos3/cli.py
When the URL named a single object instead of a prefix (head_object
hit), _scan_s3 yielded TWO FileInfos with relative_path="":

  1) FileInfo("", N, is_dir=False)  — the file
  2) FileInfo("", 0, is_dir=True)   — the unconditional root-dir marker

_compute_sync_diff builds a dict keyed only on relative_path, so the
dir marker overwrote the file. _perform_download then mkdir'd the
destination via _put_locally's is_dir branch and never called
download_file. download() returned a path to that empty directory and
the CLI exited 0 — caller's `data_file=$(pos3 download s3://b/x.json)`
got a directory where the object should have been.

Fix: in _scan_s3, track whether we already emitted a file at
relative_path="" (the exact-object case) and suppress the redundant
root marker in that case. Directory listings are unaffected — for
prefix listings _list_s3_objects appends "/" so no listed key matches
prefix exactly, has_root_file stays False, the symmetry-with-_scan_local
root marker is still emitted.

Reproduced before fix:

  >>> with mirror(...): download('s3://bucket/results.json', local=...)
  download_file called: False  # !!
  local exists: True (as a directory)

After fix:

  >>> download_file called: True
  >>> args: ('bucket', 'results.json', '/.../results.json')

Tests:
  - test_download_single_object_calls_download_file (test_s3.py): API
    level, asserts download_file called with the right (bucket, key,
    dst).
  - test_download_single_object_calls_download_file (test_cli.py): CLI
    level, asserts rc==0, download_file called, and the local path is
    printed to stdout as the success contract.

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d5e60b62d8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pos3/__init__.py Outdated
ls() normalized the input URL via _normalize_s3_url, which strips
trailing slashes. So `pos3 ls s3://bucket/data/` became a lookup of
key="data" — and if an object exactly named `data` also existed
alongside the `data/` "directory", head_object('data') hit in
_list_s3_objects and the single-object branch silently won, returning
only `s3://bucket/data` and hiding the directory contents the user
clearly asked for.

Fix: in ls(), use _parse_s3_url directly instead of normalizing first.
The trailing slash carries user intent ("treat as a directory prefix")
that _list_s3_objects already respects (it skips head_object when the
key ends in /). Other ls flows are unaffected: bucket-root (empty key)
and exact-object (no slash) both still behave the same.

Reproduced before fix:
  ls s3://bucket/data/  →  s3://bucket/data            (wrong)
After fix:
  ls s3://bucket/data/  →  s3://bucket/data/file.txt   (correct)
  ls s3://bucket/data   →  s3://bucket/data            (still correct,
                                                       single-object case)

Test: test_ls_trailing_slash_forces_directory_listing sets head_object
to return 200 (an exact 'data' object also exists) AND paginate to
return data/file.txt, then asserts ls s3://bucket/data/ yields only
the directory-content line.

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ef302d64d4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pos3/__init__.py Outdated
Same root issue as the ls fix in ef302d6, now in the plan_* methods.
After the earlier double-slash fix (8479ea5) both planners parsed the
*normalized* URL, which stripped the user's trailing slash before
_scan_s3 → _list_s3_objects. If both `data` (exact object) and `data/`
(directory) existed, head_object('data') hit the exact-key branch and
the plan reported the single object instead of the directory contents
the user asked for with the trailing slash.

But we can't simply drop the normalization — the earlier fix used it
to keep _make_s3_key from producing `data//file.txt`.

Resolution: split the prefix. Use the raw _parse_s3_url(remote) for the
S3 scan (so _list_s3_objects sees the trailing slash and skips
head_object) and the normalized _parse_s3_url(_normalize_s3_url(remote))
for output-key construction (so reconstructed URLs don't double up the
slash). _scan_s3 strips len(scan_prefix) then lstrip("/") so
FileInfo.relative_path is identical either way.

Reproduced before fix (with head_object('data') 200 AND paginate of
data/file.txt):
  download -n s3://bucket/data/   →  download: s3://bucket/data to /tmp/x
After:
  download -n s3://bucket/data/   →  download: s3://bucket/data/file.txt
                                              to /tmp/x/file.txt
  download -n s3://bucket/data    →  download: s3://bucket/data to /tmp/x
                                       (still correct — single object)

Test: test_plan_download_trailing_slash_forces_directory_listing
mocks both head_object('data') 200 and a data/file.txt in the
paginate; asserts plan.to_copy emits the directory-content URL.

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8353ac8ed2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pos3/cli.py
Comment thread pos3/__init__.py
Two related P2s flagged by codex:

1. The dry-run/plan fixes from 8353ac8 preserved the user's trailing
   slash, but the real Mirror.download → _perform_download and the
   _sync_uploads paths still normalized the URL before scanning. When
   both `data` (exact object) and `data/...` existed, head_object('data')
   won and download() pulled the wrong target — silently — into the
   destination, exiting 0.

2. plan_upload() with a missing local source returned to_delete = every
   remote object, but real _sync_uploads short-circuits any registration
   whose local_path.exists() is False (no transfers AND no deletes).
   Plan would claim "would delete all remote data" for an action that
   would in fact delete nothing.

Fixes:

- _perform_download: now uses the dual-prefix pattern (raw for scan,
  normalized for output keys). Mirror.download passes the raw `remote`
  through instead of `normalized`. Registration code is unchanged
  (registration is keyed on normalized form, which is the right
  identity for dedup).

- _UploadRegistration grows a `raw_remote` field (excluded from __eq__).
  Mirror.upload populates it from the user's input. _sync_uploads uses
  it for the dual-prefix pattern, falling back to `remote` if not set
  (defensive default of "" preserves backwards compat).

- plan_upload short-circuits to TransferPlan(to_copy=[], to_delete=[])
  when `source` does not exist — matches what real _sync_uploads would
  do for that registration.

Tests added under TestTrailingSlashRealTransfers:
  - test_download_trailing_slash_transfers_directory_contents
  - test_upload_trailing_slash_scans_directory_for_delete
  - test_plan_upload_empty_when_source_missing

Each one pretends both the exact key and the directory contents exist;
asserts the correct one is acted on (or, for plan_upload missing source,
that no action is planned at all).

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas

vertix commented May 30, 2026

Copy link
Copy Markdown
Contributor Author

@codex review


Generated by Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 64f4c6c026

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pos3/cli.py
Two related UX gaps:

1. main() only caught (ValueError, TransferError). _scan_s3 →
   _list_s3_objects calls head_object/paginate before any worker future
   can wrap a failure in TransferError, and _list_s3_objects re-raises
   non-404 ClientErrors directly. So access-denied, missing-bucket,
   throttling, or expired-credential errors escaped main() and surfaced
   as a Python traceback, contradicting the documented
     pos3 <cmd>: <message>\nexit 1
   contract that callers like `data_dir=$(pos3 download …)` rely on.

   Catch botocore's BotoCoreError and ClientError alongside the existing
   ValueError / TransferError. Message format includes the exception
   class name so the user can tell auth errors from network errors
   without a traceback.

2. argparse routes "unrecognized arguments" through the TOP-LEVEL parser's
   error(). So `pos3 download s3://b/k --dry_run` (underscore typo) gave:

     usage: pos3 [-h] {ls,download,upload} ...
     pos3: error: unrecognized arguments: --dry_run

   which doesn't show the user that download has a -n / --dry-run flag.
   They had to separately run `pos3 download --help` to find it.

   Use parse_known_args + look up the chosen subparser via the
   _SubParsersAction so the SUBCOMMAND's usage is what gets printed:

     usage: pos3 download [-h] [--local PATH] [--delete] [--exclude PATTERN] [-n]
                          [--profile NAME]
                          url
     pos3 download: error: unrecognized arguments: --dry_run

Tests:
  - TestCliBotoErrors covers ls, real download, and dry-run download
    when head_object raises ClientError(403). Each asserts rc == 1, a
    pos3 <cmd>: prefix on stderr, and (for download) captured.out == "".
  - TestCliEntry.test_unknown_flag_uses_subcommand_usage asserts the
    subcommand usage line appears and the top-level
    `{ls,download,upload}` group does not.
  - test_upload_uploads_existing_local_source extended to assert
    captured.out == "" — the upload counterpart to download's "exactly
    one line = local path on stdout" contract.

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7c32fa24fa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pos3/__init__.py
Codex's P2: I'd advertised Mirror.plan_download / plan_upload as
public API in the README and CHANGELOG, but they only lived on the
private _Mirror class. Module-level wrappers existed for download,
upload, sync, and ls, but not for the planning entry points; neither
TransferPlan nor TransferError were in __all__. Users following the
README would have had to reach into pos3._require_active_mirror() —
which is itself underscored.

Fix the surface, not just the docs:

- Add module-level pos3.plan_download(remote, ...) and
  pos3.plan_upload(remote, ...) wrappers, same shape as the existing
  pos3.download / pos3.upload (require an active mirror context, delegate
  to the active Mirror instance).
- Expand __all__ to include plan_download, plan_upload, TransferError,
  and TransferPlan.
- CLI now imports and calls these module-level wrappers (dogfood).
- README's dry-run section gets a Python example using
  pos3.plan_download, no longer references the private Mirror class.
- CHANGELOG entry updated accordingly.

TestPlanPublicAPI adds two tests: one asserting callability and __all__
membership for both wrappers, TransferPlan, and TransferError; another
exercising pos3.plan_download end-to-end via the public API.

Full suite: 136 passed.

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b3e7a840e2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pos3/__init__.py Outdated
The earlier fix for sync_on_error cleanup masking caught only
TransferError, but _sync_uploads can fail before any worker future
exists — _scan_s3 → _list_s3_objects calls head_object/paginate
first, and a non-404 ClientError (403, expired creds, throttling,
missing bucket) propagates straight through the scan iterator. So
cleanup-time S3 scan failures would still replace the user's
application exception, which is exactly the regression the
had_error=True path was supposed to avoid.

The had_error=True path is fundamentally "best-effort, do no more
harm" — anything that escapes _sync_uploads during cleanup must be
logged and swallowed so the original exception stays the visible
cause. Broaden the catch to Exception. Comment in the source spells
out the failure modes (TransferError from workers, ClientError /
BotoCoreError from the pre-worker scan, OSError on the local fs).

Clean-exit path (had_error=False) is unchanged — still propagates so
one-shot CLI / library callers see definitive failures.

Test: test_app_exception_survives_scan_client_error_in_cleanup sets
mock_s3.head_object.side_effect = ClientError(403), registers a
sync_on_error=True upload, raises AppError inside the mirror() body,
and asserts AppError propagates (not ClientError). Pre-fix this would
have failed.

https://claude.ai/code/session_01QoQaZ2G6FsNpSZyStqTXas

vertix commented May 30, 2026

Copy link
Copy Markdown
Contributor Author

@codex review


Generated by Claude Code

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@vertix vertix merged commit fcdae81 into main May 30, 2026
4 checks passed
@vertix vertix deleted the claude/add-documentation-trVyZ branch May 30, 2026 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants