From 7ed945002060ad1a40bb034a70b7d4942229b095 Mon Sep 17 00:00:00 2001 From: skulidropek <66840575+skulidropek@users.noreply.github.com> Date: Fri, 22 May 2026 06:55:39 +0000 Subject: [PATCH 1/4] docs(spec): add ABC-5 clone cache plan --- .kanban/changes/ABC-5/design.md | 66 +++++++++++++++++++++++++++++++ .kanban/changes/ABC-5/proposal.md | 44 +++++++++++++++++++++ .kanban/changes/ABC-5/tasks.md | 39 ++++++++++++++++++ 3 files changed, 149 insertions(+) create mode 100644 .kanban/changes/ABC-5/design.md create mode 100644 .kanban/changes/ABC-5/proposal.md create mode 100644 .kanban/changes/ABC-5/tasks.md diff --git a/.kanban/changes/ABC-5/design.md b/.kanban/changes/ABC-5/design.md new file mode 100644 index 00000000..9f3fd164 --- /dev/null +++ b/.kanban/changes/ABC-5/design.md @@ -0,0 +1,66 @@ +# ABC-5 Design: Repository Download Cache + +## Architecture Boundary + +The change belongs in the generated container entrypoint clone shell, rendered by pure TypeScript template functions: + +- Source template: `packages/lib/src/core/templates-entrypoint/tasks.ts` +- App mirror: `packages/app/src/lib/core/templates-entrypoint/tasks.ts` +- Compose cache mount: `packages/lib/src/core/templates/docker-compose.ts` and app mirror + +The TypeScript layer remains pure template generation. Git, filesystem, and network operations remain in the generated shell inside the project container. + +## Proposed Flow + +1. Resolve `AUTH_REPO_URL` using the existing auth-label logic. +2. Resolve deterministic cache key from canonical `REPO_URL`. +3. Ensure cache root exists under `/home//.docker-git/.cache`. +4. If a valid cache exists, refresh it before workspace preparation. +5. Prepare `TARGET_DIR` from cached data. +6. Checkout or create the branch requested by `REPO_REF`. +7. Set remotes using the existing fork/upstream logic. +8. Mark clone completion with the existing `/run/docker-git/clone.done` or `/run/docker-git/clone.failed` markers. + +## Cache Shape + +Use the existing bare mirror cache unless implementation proves a working-copy cache is required for the issue semantics. If a working-copy cache is introduced, keep it in a distinct path such as: + +- `/home//.docker-git/.cache/git-worktrees/` + +The bare mirror path remains: + +- `/home//.docker-git/.cache/git-mirrors/.git` + +## Invariants + +- `forall repoUrl: cacheKey(repoUrl)` is deterministic and does not include tokens. +- `forall clone: cache refresh happens before target workspace checkout when a cache path exists`. +- `forall target: clone_ok(target) -> target/.git exists`. +- `forall target, repoRef: clone_ok(target, repoRef) -> target HEAD is repoRef-compatible`. +- `forall refreshedRefs: refreshedRefs subset refs/heads/* union refs/tags/*` for broad mirror refresh. +- `forall cache paths: cache path is under ~/.docker-git/.cache and ignored by state repo`. +- Authenticated URLs may be used for network commands, but tokenized URLs must not become cache keys or persisted remotes. + +## Branch Handling + +- Plain branch refs: checkout the requested branch from refreshed cache/remote. +- Issue refs such as `issue-138`: clone default branch, then create or reset local issue branch as existing behavior does. +- GitHub PR refs `refs/pull//head`: fetch the specific PR ref into deterministic local branch `pr-`. +- GitLab MR refs `refs/merge-requests//head`: fetch the specific MR ref into deterministic local branch `mr-`. +- Empty `REPO_REF`: use the remote default branch. + +## Failure Behavior + +- Invalid cache paths are removed or ignored and the flow falls back to network clone. +- Cache refresh failure should not corrupt an existing valid cache. +- If final workspace checkout fails, set `CLONE_OK=0` and write the existing failure marker. + +## Risks + +- Concurrent clones of the same repository may race while refreshing or creating cache paths. +- Working-copy caches need locking if added. +- `git pull` semantics differ for branch refs, issue branches, and PR/MR refs; implementation should prefer explicit `fetch` plus checkout/reset where that is more deterministic. + +## Recommended Implementation Constraint + +Prefer minimal changes to the existing bare mirror flow first. Treat "git pull под нужную нам ветку" as the externally visible invariant: cache is refreshed, then workspace is checked out to the requested branch. Add a working-copy cache only if e2e verification shows the bare mirror flow cannot satisfy the requirement. diff --git a/.kanban/changes/ABC-5/proposal.md b/.kanban/changes/ABC-5/proposal.md new file mode 100644 index 00000000..2f17b385 --- /dev/null +++ b/.kanban/changes/ABC-5/proposal.md @@ -0,0 +1,44 @@ +# ABC-5 Proposal: Repository Download Cache + +## Summary + +Implement repository download caching for `docker-git clone` so repeated clones of the same repository reuse shared cached git data and refresh that cache before preparing the requested workspace branch. + +## Requirement + +GitHub issue #138: + +> Что бы один и тот же репозиторий лежал бы в кеше и мы бы грузили данные из кеша + git pull под нужную нам ветку + +## Current State + +- The clone flow is generated as shell from `packages/lib/src/core/templates-entrypoint/tasks.ts` and mirrored in `packages/app/src/lib/core/templates-entrypoint/tasks.ts`. +- A shared Docker volume already mounts `/home/dev/.docker-git/.cache` into project containers. +- Existing cache behavior stores a per-`REPO_URL` bare mirror under `/home/dev/.docker-git/.cache/git-mirrors`. +- Existing clone behavior uses `--reference-if-able` and `--dissociate`, then performs branch or PR/MR checkout logic in the target workspace. +- Existing behavior refreshes the bare mirror with `git fetch --prune`, but it does not provide a cached working repository that is updated via branch-aware pull before workspace preparation. + +## Scope + +- Add or refine clone-cache behavior inside the generated entrypoint clone task. +- Keep cache state inside the shared cache volume. +- Preserve existing parsing of `REPO_URL`, `REPO_REF`, issue URLs, PR refs, GitLab merge request refs, auth labels, and fork remotes. +- Keep state repository sync behavior separate from repository download caching. +- Extend tests around template rendering and clone-cache e2e behavior. + +## Non-Goals + +- Do not change CLI argument semantics. +- Do not merge work into `main`. +- Do not cache secrets or auth material in repository cache paths. +- Do not commit cache artifacts into the `.docker-git` state repository. +- Do not broaden mirror refresh to hosted forge PR/MR refs. + +## Acceptance Criteria + +- Repeated clones of the same `REPO_URL` use the same cache key/path. +- The cache is refreshed before it is used for a workspace. +- The resulting workspace is checked out to the requested `REPO_REF` or deterministic local issue/PR/MR branch. +- Cache paths remain ignored and untracked by state repository sync. +- Existing clone flows continue to work for normal branches, issue URLs, GitHub PR refs, GitLab MR refs, and no-ref clones. +- Tests document the cache reuse and branch-refresh contract. diff --git a/.kanban/changes/ABC-5/tasks.md b/.kanban/changes/ABC-5/tasks.md new file mode 100644 index 00000000..b5fa8610 --- /dev/null +++ b/.kanban/changes/ABC-5/tasks.md @@ -0,0 +1,39 @@ +# ABC-5 Tasks + +## Implementation + +- [ ] Update `packages/lib/src/core/templates-entrypoint/tasks.ts` clone-cache flow. +- [ ] Apply the same template update to `packages/app/src/lib/core/templates-entrypoint/tasks.ts`. +- [ ] Keep cache roots under `/home//.docker-git/.cache`. +- [ ] Preserve existing auth-label and token handling. +- [ ] Preserve existing fork remote behavior after clone. +- [ ] Preserve existing clone completion markers. +- [ ] Ensure cache paths are ignored by state repo sync if new cache directories are added. + +## Tests + +- [ ] Add or extend template tests for cache initialization, refresh, and reuse markers. +- [ ] Add or extend template tests for branch, issue branch, GitHub PR ref, and GitLab MR ref behavior. +- [ ] Extend `scripts/e2e/clone-cache.sh` to verify same repository cache reuse across two clones. +- [ ] Verify second clone uses cache and ends on the requested branch. +- [ ] Verify cache artifacts are not tracked by the state repository. + +## Verification Commands + +- [ ] `bun run --filter @effect-template/lib test` +- [ ] `bun run --filter @prover-coder-ai/docker-git test` +- [ ] `bun run typecheck` +- [ ] `bun run e2e:clone-cache` + +## Out of Scope + +- [ ] No CLI flag changes. +- [ ] No API contract changes. +- [ ] No state repo auto-pull changes. +- [ ] No merge to `main`. + +## Open Questions + +- Should the final implementation use only the existing bare mirror cache, or introduce a separate working-copy cache to match the literal "git pull" wording? +- Should concurrent clone cache refreshes use a lock file in the shared cache volume? +- Should failed cache refresh be a warning with fallback or a hard clone failure? From 55ffaa3b73918c1ca62aa81a04ddb067e8d795ad Mon Sep 17 00:00:00 2001 From: skulidropek <66840575+skulidropek@users.noreply.github.com> Date: Fri, 22 May 2026 07:10:52 +0000 Subject: [PATCH 2/4] feat(clone): reuse repository cache for branch pulls --- .kanban/changes/ABC-5/tasks.md | 32 +++++++------- .../lib/core/templates-entrypoint/tasks.ts | 28 +++++++++--- .../src/core/templates-entrypoint/tasks.ts | 28 +++++++++--- packages/lib/tests/core/templates.test.ts | 44 +++++++++++++++++++ scripts/e2e/clone-cache.sh | 8 ++-- 5 files changed, 111 insertions(+), 29 deletions(-) diff --git a/.kanban/changes/ABC-5/tasks.md b/.kanban/changes/ABC-5/tasks.md index b5fa8610..b795efe0 100644 --- a/.kanban/changes/ABC-5/tasks.md +++ b/.kanban/changes/ABC-5/tasks.md @@ -2,28 +2,28 @@ ## Implementation -- [ ] Update `packages/lib/src/core/templates-entrypoint/tasks.ts` clone-cache flow. -- [ ] Apply the same template update to `packages/app/src/lib/core/templates-entrypoint/tasks.ts`. -- [ ] Keep cache roots under `/home//.docker-git/.cache`. -- [ ] Preserve existing auth-label and token handling. -- [ ] Preserve existing fork remote behavior after clone. -- [ ] Preserve existing clone completion markers. -- [ ] Ensure cache paths are ignored by state repo sync if new cache directories are added. +- [x] Update `packages/lib/src/core/templates-entrypoint/tasks.ts` clone-cache flow. +- [x] Apply the same template update to `packages/app/src/lib/core/templates-entrypoint/tasks.ts`. +- [x] Keep cache roots under `/home//.docker-git/.cache`. +- [x] Preserve existing auth-label and token handling. +- [x] Preserve existing fork remote behavior after clone. +- [x] Preserve existing clone completion markers. +- [x] Ensure cache paths are ignored by state repo sync if new cache directories are added. ## Tests -- [ ] Add or extend template tests for cache initialization, refresh, and reuse markers. -- [ ] Add or extend template tests for branch, issue branch, GitHub PR ref, and GitLab MR ref behavior. -- [ ] Extend `scripts/e2e/clone-cache.sh` to verify same repository cache reuse across two clones. -- [ ] Verify second clone uses cache and ends on the requested branch. -- [ ] Verify cache artifacts are not tracked by the state repository. +- [x] Add or extend template tests for cache initialization, refresh, and reuse markers. +- [x] Add or extend template tests for branch, issue branch, GitHub PR ref, and GitLab MR ref behavior. +- [x] Extend `scripts/e2e/clone-cache.sh` to verify same repository cache reuse across two clones. +- [x] Verify second clone uses cache and ends on the requested branch. +- [x] Verify cache artifacts are not tracked by the state repository. ## Verification Commands -- [ ] `bun run --filter @effect-template/lib test` -- [ ] `bun run --filter @prover-coder-ai/docker-git test` -- [ ] `bun run typecheck` -- [ ] `bun run e2e:clone-cache` +- [x] `bun run --filter @effect-template/lib test` +- [x] `bun run --filter @prover-coder-ai/docker-git test` +- [x] `bun run typecheck` +- [ ] `bun run e2e:clone-cache` (blocked locally: docker-git host CLI cannot auto-discover controller over remote `DOCKER_HOST=tcp://host.docker.internal:2375`) ## Out of Scope diff --git a/packages/app/src/lib/core/templates-entrypoint/tasks.ts b/packages/app/src/lib/core/templates-entrypoint/tasks.ts index 1889eb05..702433fd 100644 --- a/packages/app/src/lib/core/templates-entrypoint/tasks.ts +++ b/packages/app/src/lib/core/templates-entrypoint/tasks.ts @@ -130,6 +130,8 @@ const cloneCacheRefreshRefspecs = "'+refs/heads/*:refs/heads/*' '+refs/tags/*:re const renderCloneCacheInit = (config: TemplateConfig): string => ` CLONE_CACHE_ARGS="" + CLONE_SOURCE_REPO="$AUTH_REPO_URL" + CLONE_USED_CACHE=0 CACHE_REPO_DIR="" CACHE_ROOT="/home/${config.sshUser}/.docker-git/.cache/git-mirrors" if command -v sha256sum >/dev/null 2>&1; then @@ -150,6 +152,8 @@ const renderCloneCacheInit = (config: TemplateConfig): string => echo "[clone-cache] mirror refresh failed for $REPO_URL" fi CLONE_CACHE_ARGS="--reference-if-able '$CACHE_REPO_DIR' --dissociate" + CLONE_SOURCE_REPO="$CACHE_REPO_DIR" + CLONE_USED_CACHE=1 echo "[clone-cache] using mirror: $CACHE_REPO_DIR" else echo "[clone-cache] invalid mirror removed: $CACHE_REPO_DIR" @@ -170,19 +174,19 @@ const renderCloneBodyRef = (config: TemplateConfig): string => String.raw` if [[ -n "$REPO_REF" ]]; then if [[ "$REPO_REF" == refs/pull/* || "$REPO_REF" == refs/merge-requests/* ]]; then REF_BRANCH="$(printf "%s" "$REPO_REF" | sed -E 's#^refs/pull/([^/]+)/head$#pr-\1#; s#^refs/merge-requests/([^/]+)/head$#mr-\1#')" - if ! su - ${config.sshUser} -c "GIT_TERMINAL_PROMPT=0 git clone --progress $CLONE_CACHE_ARGS '$AUTH_REPO_URL' '$TARGET_DIR'"; then + if ! su - ${config.sshUser} -c "GIT_TERMINAL_PROMPT=0 git clone --progress $CLONE_CACHE_ARGS '$CLONE_SOURCE_REPO' '$TARGET_DIR'"; then echo "[clone] git clone failed for $REPO_URL" CLONE_OK=0 else - if ! su - ${config.sshUser} -c "cd '$TARGET_DIR' && GIT_TERMINAL_PROMPT=0 git fetch --progress origin '$REPO_REF':'$REF_BRANCH' && git checkout '$REF_BRANCH'"; then + if ! su - ${config.sshUser} -c "cd '$TARGET_DIR' && git remote set-url origin '$AUTH_REPO_URL' && GIT_TERMINAL_PROMPT=0 git fetch --progress origin '$REPO_REF':'$REF_BRANCH' && git checkout '$REF_BRANCH'"; then echo "[clone] git fetch failed for $REPO_REF" CLONE_OK=0 fi fi else - if ! su - ${config.sshUser} -c "GIT_TERMINAL_PROMPT=0 git clone --progress $CLONE_CACHE_ARGS --branch '$REPO_REF' '$AUTH_REPO_URL' '$TARGET_DIR'"; then + if ! su - ${config.sshUser} -c "GIT_TERMINAL_PROMPT=0 git clone --progress $CLONE_CACHE_ARGS --branch '$REPO_REF' '$CLONE_SOURCE_REPO' '$TARGET_DIR'"; then echo "[clone] branch '$REPO_REF' missing; retrying without --branch" - if ! su - ${config.sshUser} -c "GIT_TERMINAL_PROMPT=0 git clone --progress $CLONE_CACHE_ARGS '$AUTH_REPO_URL' '$TARGET_DIR'"; then + if ! su - ${config.sshUser} -c "GIT_TERMINAL_PROMPT=0 git clone --progress $CLONE_CACHE_ARGS '$CLONE_SOURCE_REPO' '$TARGET_DIR'"; then echo "[clone] git clone failed for $REPO_URL" CLONE_OK=0 elif [[ "$REPO_REF" == issue-* ]]; then @@ -191,12 +195,26 @@ const renderCloneBodyRef = (config: TemplateConfig): string => CLONE_OK=0 fi fi + elif [[ "$CLONE_USED_CACHE" == "1" ]]; then + if ! su - ${config.sshUser} -c "cd '$TARGET_DIR' && git remote set-url origin '$AUTH_REPO_URL' && GIT_TERMINAL_PROMPT=0 git pull --ff-only origin '$REPO_REF'"; then + echo "[clone-cache] git pull failed for $REPO_REF" + CLONE_OK=0 + else + echo "[clone-cache] pulled branch: $REPO_REF" + fi fi fi else - if ! su - ${config.sshUser} -c "GIT_TERMINAL_PROMPT=0 git clone --progress $CLONE_CACHE_ARGS '$AUTH_REPO_URL' '$TARGET_DIR'"; then + if ! su - ${config.sshUser} -c "GIT_TERMINAL_PROMPT=0 git clone --progress $CLONE_CACHE_ARGS '$CLONE_SOURCE_REPO' '$TARGET_DIR'"; then echo "[clone] git clone failed for $REPO_URL" CLONE_OK=0 + elif [[ "$CLONE_USED_CACHE" == "1" ]]; then + if ! su - ${config.sshUser} -c "cd '$TARGET_DIR' && git remote set-url origin '$AUTH_REPO_URL' && GIT_TERMINAL_PROMPT=0 git pull --ff-only"; then + echo "[clone-cache] git pull failed for default branch" + CLONE_OK=0 + else + echo "[clone-cache] pulled default branch" + fi fi fi` diff --git a/packages/lib/src/core/templates-entrypoint/tasks.ts b/packages/lib/src/core/templates-entrypoint/tasks.ts index 1889eb05..702433fd 100644 --- a/packages/lib/src/core/templates-entrypoint/tasks.ts +++ b/packages/lib/src/core/templates-entrypoint/tasks.ts @@ -130,6 +130,8 @@ const cloneCacheRefreshRefspecs = "'+refs/heads/*:refs/heads/*' '+refs/tags/*:re const renderCloneCacheInit = (config: TemplateConfig): string => ` CLONE_CACHE_ARGS="" + CLONE_SOURCE_REPO="$AUTH_REPO_URL" + CLONE_USED_CACHE=0 CACHE_REPO_DIR="" CACHE_ROOT="/home/${config.sshUser}/.docker-git/.cache/git-mirrors" if command -v sha256sum >/dev/null 2>&1; then @@ -150,6 +152,8 @@ const renderCloneCacheInit = (config: TemplateConfig): string => echo "[clone-cache] mirror refresh failed for $REPO_URL" fi CLONE_CACHE_ARGS="--reference-if-able '$CACHE_REPO_DIR' --dissociate" + CLONE_SOURCE_REPO="$CACHE_REPO_DIR" + CLONE_USED_CACHE=1 echo "[clone-cache] using mirror: $CACHE_REPO_DIR" else echo "[clone-cache] invalid mirror removed: $CACHE_REPO_DIR" @@ -170,19 +174,19 @@ const renderCloneBodyRef = (config: TemplateConfig): string => String.raw` if [[ -n "$REPO_REF" ]]; then if [[ "$REPO_REF" == refs/pull/* || "$REPO_REF" == refs/merge-requests/* ]]; then REF_BRANCH="$(printf "%s" "$REPO_REF" | sed -E 's#^refs/pull/([^/]+)/head$#pr-\1#; s#^refs/merge-requests/([^/]+)/head$#mr-\1#')" - if ! su - ${config.sshUser} -c "GIT_TERMINAL_PROMPT=0 git clone --progress $CLONE_CACHE_ARGS '$AUTH_REPO_URL' '$TARGET_DIR'"; then + if ! su - ${config.sshUser} -c "GIT_TERMINAL_PROMPT=0 git clone --progress $CLONE_CACHE_ARGS '$CLONE_SOURCE_REPO' '$TARGET_DIR'"; then echo "[clone] git clone failed for $REPO_URL" CLONE_OK=0 else - if ! su - ${config.sshUser} -c "cd '$TARGET_DIR' && GIT_TERMINAL_PROMPT=0 git fetch --progress origin '$REPO_REF':'$REF_BRANCH' && git checkout '$REF_BRANCH'"; then + if ! su - ${config.sshUser} -c "cd '$TARGET_DIR' && git remote set-url origin '$AUTH_REPO_URL' && GIT_TERMINAL_PROMPT=0 git fetch --progress origin '$REPO_REF':'$REF_BRANCH' && git checkout '$REF_BRANCH'"; then echo "[clone] git fetch failed for $REPO_REF" CLONE_OK=0 fi fi else - if ! su - ${config.sshUser} -c "GIT_TERMINAL_PROMPT=0 git clone --progress $CLONE_CACHE_ARGS --branch '$REPO_REF' '$AUTH_REPO_URL' '$TARGET_DIR'"; then + if ! su - ${config.sshUser} -c "GIT_TERMINAL_PROMPT=0 git clone --progress $CLONE_CACHE_ARGS --branch '$REPO_REF' '$CLONE_SOURCE_REPO' '$TARGET_DIR'"; then echo "[clone] branch '$REPO_REF' missing; retrying without --branch" - if ! su - ${config.sshUser} -c "GIT_TERMINAL_PROMPT=0 git clone --progress $CLONE_CACHE_ARGS '$AUTH_REPO_URL' '$TARGET_DIR'"; then + if ! su - ${config.sshUser} -c "GIT_TERMINAL_PROMPT=0 git clone --progress $CLONE_CACHE_ARGS '$CLONE_SOURCE_REPO' '$TARGET_DIR'"; then echo "[clone] git clone failed for $REPO_URL" CLONE_OK=0 elif [[ "$REPO_REF" == issue-* ]]; then @@ -191,12 +195,26 @@ const renderCloneBodyRef = (config: TemplateConfig): string => CLONE_OK=0 fi fi + elif [[ "$CLONE_USED_CACHE" == "1" ]]; then + if ! su - ${config.sshUser} -c "cd '$TARGET_DIR' && git remote set-url origin '$AUTH_REPO_URL' && GIT_TERMINAL_PROMPT=0 git pull --ff-only origin '$REPO_REF'"; then + echo "[clone-cache] git pull failed for $REPO_REF" + CLONE_OK=0 + else + echo "[clone-cache] pulled branch: $REPO_REF" + fi fi fi else - if ! su - ${config.sshUser} -c "GIT_TERMINAL_PROMPT=0 git clone --progress $CLONE_CACHE_ARGS '$AUTH_REPO_URL' '$TARGET_DIR'"; then + if ! su - ${config.sshUser} -c "GIT_TERMINAL_PROMPT=0 git clone --progress $CLONE_CACHE_ARGS '$CLONE_SOURCE_REPO' '$TARGET_DIR'"; then echo "[clone] git clone failed for $REPO_URL" CLONE_OK=0 + elif [[ "$CLONE_USED_CACHE" == "1" ]]; then + if ! su - ${config.sshUser} -c "cd '$TARGET_DIR' && git remote set-url origin '$AUTH_REPO_URL' && GIT_TERMINAL_PROMPT=0 git pull --ff-only"; then + echo "[clone-cache] git pull failed for default branch" + CLONE_OK=0 + else + echo "[clone-cache] pulled default branch" + fi fi fi` diff --git a/packages/lib/tests/core/templates.test.ts b/packages/lib/tests/core/templates.test.ts index deb35533..b67ca83c 100644 --- a/packages/lib/tests/core/templates.test.ts +++ b/packages/lib/tests/core/templates.test.ts @@ -306,6 +306,50 @@ describe("renderEntrypoint clone cache", () => { expect(entrypoint).not.toContain("'+refs/merge-requests/*:refs/merge-requests/*'") }) + it("uses refreshed mirrors as clone source and pulls cached branch clones", () => { + const entrypoint = renderEntrypoint(makeTemplateConfig()) + + expectContainsAll(entrypoint, [ + 'CLONE_SOURCE_REPO="$AUTH_REPO_URL"', + "CLONE_SOURCE_REPO=\"$CACHE_REPO_DIR\"", + "CLONE_USED_CACHE=1", + "git clone --progress $CLONE_CACHE_ARGS --branch '$REPO_REF' '$CLONE_SOURCE_REPO' '$TARGET_DIR'", + "git remote set-url origin '$AUTH_REPO_URL' && GIT_TERMINAL_PROMPT=0 git pull --ff-only origin '$REPO_REF'", + 'echo "[clone-cache] pulled branch: $REPO_REF"' + ]) + }) + + it("keeps issue refs as local branches after fallback clone", () => { + const entrypoint = renderEntrypoint(makeTemplateConfig()) + + expectContainsAll(entrypoint, [ + 'elif [[ "$REPO_REF" == issue-* ]]; then', + "cd '$TARGET_DIR' && git checkout -B '$REPO_REF'", + 'echo "[clone] failed to create local branch \'$REPO_REF\'"' + ]) + }) + + it("keeps PR and merge request refs as explicit fetches after cached clone", () => { + const entrypoint = renderEntrypoint(makeTemplateConfig()) + + expect(entrypoint).toContain( + "git clone --progress $CLONE_CACHE_ARGS '$CLONE_SOURCE_REPO' '$TARGET_DIR'" + ) + expect(entrypoint).toContain( + "git remote set-url origin '$AUTH_REPO_URL' && GIT_TERMINAL_PROMPT=0 git fetch --progress origin '$REPO_REF':'$REF_BRANCH' && git checkout '$REF_BRANCH'" + ) + }) + + it("pulls the remote default branch when cloning from a warm cache without repoRef", () => { + const entrypoint = renderEntrypoint(makeTemplateConfig()) + + expectContainsAll(entrypoint, [ + "git clone --progress $CLONE_CACHE_ARGS '$CLONE_SOURCE_REPO' '$TARGET_DIR'", + "git remote set-url origin '$AUTH_REPO_URL' && GIT_TERMINAL_PROMPT=0 git pull --ff-only", + 'echo "[clone-cache] pulled default branch"' + ]) + }) + it("preserves branch/tag-only clone-cache refspecs for generated configs", () => { fc.assert( fc.property(generatedTemplateConfigArbitrary, (config) => { diff --git a/scripts/e2e/clone-cache.sh b/scripts/e2e/clone-cache.sh index dcf6bffa..fcc6cc07 100755 --- a/scripts/e2e/clone-cache.sh +++ b/scripts/e2e/clone-cache.sh @@ -26,8 +26,8 @@ export DOCKER_GIT_PROJECTS_ROOT="$ROOT" export DOCKER_GIT_STATE_AUTO_PULL=0 export DOCKER_GIT_STATE_AUTO_SYNC=0 -REPO_URL="https://github.com/octocat/Hello-World/issues/1" -TARGET_DIR="/home/dev/workspaces/octocat/hello-world/issue-1" +REPO_URL="https://github.com/octocat/Hello-World/tree/master" +TARGET_DIR="/home/dev/workspaces/octocat/hello-world" MIRROR_PREFIX="/home/dev/.docker-git/.cache/git-mirrors" ACTIVE_OUT_DIR="" @@ -174,7 +174,7 @@ EOF_ENV local branch branch="$(dg_project_docker exec -u dev "$container_name" bash -lc "cd '$TARGET_DIR' && git rev-parse --abbrev-ref HEAD")" - [[ "$branch" == "issue-1" ]] || fail "expected branch issue-1, got: $branch" + [[ "$branch" == "master" ]] || fail "expected branch master, got: $branch" if [[ "$expect_cache_use" == "1" ]]; then if [[ -n "$expected_mirror_name" ]]; then @@ -184,6 +184,8 @@ EOF_ENV grep -Fq -- "[clone-cache] using mirror: $MIRROR_PREFIX/" "$log_path" \ || fail "expected cache reuse log in second clone" fi + grep -Fq -- "[clone-cache] pulled branch: master" "$log_path" \ + || fail "expected branch pull from warm cache in second clone" else grep -Fq -- "[clone-cache] mirror created: $MIRROR_PREFIX/" "$log_path" \ || grep -Fq -- "[clone-cache] using mirror: $MIRROR_PREFIX/" "$log_path" \ From 67ed16ec6b543492c80e1e5041bdbf55a934c81c Mon Sep 17 00:00:00 2001 From: skulidropek <66840575+skulidropek@users.noreply.github.com> Date: Fri, 22 May 2026 07:20:58 +0000 Subject: [PATCH 3/4] docs(spec): archive ABC-5 clone cache work --- .kanban/changes/ABC-5/design.md | 21 +++++++++++++++++++++ .kanban/changes/ABC-5/proposal.md | 8 ++++++++ .kanban/changes/ABC-5/tasks.md | 24 +++++++++++++++++++++--- 3 files changed, 50 insertions(+), 3 deletions(-) diff --git a/.kanban/changes/ABC-5/design.md b/.kanban/changes/ABC-5/design.md index 9f3fd164..c37047e2 100644 --- a/.kanban/changes/ABC-5/design.md +++ b/.kanban/changes/ABC-5/design.md @@ -64,3 +64,24 @@ The bare mirror path remains: ## Recommended Implementation Constraint Prefer minimal changes to the existing bare mirror flow first. Treat "git pull под нужную нам ветку" as the externally visible invariant: cache is refreshed, then workspace is checked out to the requested branch. Add a working-copy cache only if e2e verification shows the bare mirror flow cannot satisfy the requirement. + +## Final Design Decision + +The implementation kept the existing bare mirror cache and did not introduce a separate working-copy cache. This preserved the established cache root and state-repo ignore contract while satisfying the branch-refresh invariant: + +1. Refresh `/home//.docker-git/.cache/git-mirrors/.git` with branch/tag-only refspecs. +2. Clone warm-cache workspaces from that local mirror. +3. Restore `origin` to the authenticated repository URL before network fetch/pull. +4. Run `git pull --ff-only origin ` for normal branch refs, or `git pull --ff-only` for no-ref default-branch clones. +5. Keep PR/MR refs on explicit fetch into deterministic local branches. +6. Keep issue refs as deterministic local branches after fallback clone. + +No new cache directory was added, so the existing `.cache/git-mirrors/` ignore and untrack behavior remains sufficient. + +## Archive Invariants + +- Cache key remains derived from canonical `REPO_URL`, not `AUTH_REPO_URL`. +- Broad cache refresh remains limited to `refs/heads/*` and `refs/tags/*`. +- Tokenized auth URLs are used for network operations only; final remote normalization still runs through the existing fork/upstream remote block. +- Clone completion/failure markers remain `/run/docker-git/clone.done` and `/run/docker-git/clone.failed`. +- CLI/API contracts are unchanged. diff --git a/.kanban/changes/ABC-5/proposal.md b/.kanban/changes/ABC-5/proposal.md index 2f17b385..fff2cc17 100644 --- a/.kanban/changes/ABC-5/proposal.md +++ b/.kanban/changes/ABC-5/proposal.md @@ -42,3 +42,11 @@ GitHub issue #138: - Cache paths remain ignored and untracked by state repository sync. - Existing clone flows continue to work for normal branches, issue URLs, GitHub PR refs, GitLab MR refs, and no-ref clones. - Tests document the cache reuse and branch-refresh contract. + +## Archive Status + +- Status: implemented +- Spec commit: `f961baa docs(spec): add ABC-5 clone cache plan` +- Implementation commit: `a47e794 feat(clone): reuse repository cache for branch pulls` +- Final behavior: warm-cache clones use the refreshed bare mirror as the local clone source, then restore the authenticated origin and run branch-aware `git pull --ff-only` for normal branch/default branch flows. +- E2E note: `bun run e2e:clone-cache` is updated but could not run in the local review environment because the host CLI cannot auto-discover the controller when `DOCKER_HOST=tcp://host.docker.internal:2375` and `DOCKER_GIT_API_URL` is unset. diff --git a/.kanban/changes/ABC-5/tasks.md b/.kanban/changes/ABC-5/tasks.md index b795efe0..5a1299af 100644 --- a/.kanban/changes/ABC-5/tasks.md +++ b/.kanban/changes/ABC-5/tasks.md @@ -34,6 +34,24 @@ ## Open Questions -- Should the final implementation use only the existing bare mirror cache, or introduce a separate working-copy cache to match the literal "git pull" wording? -- Should concurrent clone cache refreshes use a lock file in the shared cache volume? -- Should failed cache refresh be a warning with fallback or a hard clone failure? +- Resolved: use the existing bare mirror cache; no separate working-copy cache was added. +- Deferred: concurrent clone cache refreshes still do not use a lock file. +- Preserved: failed cache refresh remains a warning/fallback behavior rather than a hard clone failure. + +## Archive Verification + +- [x] `bun run --filter @effect-template/lib test -- tests/core/templates.test.ts` (48 tests passed) +- [x] `bun run --filter @effect-template/lib test` (51 files, 268 tests passed) +- [x] `bun run --filter @prover-coder-ai/docker-git test` (73 files, 463 tests passed) +- [x] `bun run typecheck` +- [x] `bun run lint` +- [x] `bun run build` +- [x] Local git repro for warm-cache branch path: bare mirror refresh, clone from mirror, restore origin, `git pull --ff-only origin `. +- [ ] `bun run e2e:clone-cache` blocked by local Docker/controller discovery: `DOCKER_HOST=tcp://host.docker.internal:2375` with no reachable `DOCKER_GIT_API_URL`. + +## Archive Result + +- Completed branch: `vk/d6b8-abc-5-github-138` +- Spec commit: `f961baa docs(spec): add ABC-5 clone cache plan` +- Implementation commit: `a47e794 feat(clone): reuse repository cache for branch pulls` +- Merge status: not merged to `main`. From 49bd7a5b90e70c9362b10ff3abb24e107fa46ddf Mon Sep 17 00:00:00 2001 From: skulidropek <66840575+skulidropek@users.noreply.github.com> Date: Fri, 22 May 2026 07:42:11 +0000 Subject: [PATCH 4/4] docs(spec): refresh ABC-5 archive trail --- .kanban/changes/ABC-5/design.md | 8 ++++++++ .kanban/changes/ABC-5/proposal.md | 9 +++++++-- .kanban/changes/ABC-5/tasks.md | 18 ++++++++++++------ 3 files changed, 27 insertions(+), 8 deletions(-) diff --git a/.kanban/changes/ABC-5/design.md b/.kanban/changes/ABC-5/design.md index c37047e2..8312f056 100644 --- a/.kanban/changes/ABC-5/design.md +++ b/.kanban/changes/ABC-5/design.md @@ -85,3 +85,11 @@ No new cache directory was added, so the existing `.cache/git-mirrors/` ignore a - Tokenized auth URLs are used for network operations only; final remote normalization still runs through the existing fork/upstream remote block. - Clone completion/failure markers remain `/run/docker-git/clone.done` and `/run/docker-git/clone.failed`. - CLI/API contracts are unchanged. + +## Archive Verification Notes + +- The final implementation was rebased onto `origin/main`; implementation/archive baseline was pushed at `67ed16ec6b543492c80e1e5041bdbf55a934c81c`. +- Task-scope lint passed for both changed template files. +- Unit, package test, typecheck, and build verification passed locally. +- Full repository lint is not treated as an ABC-5 regression because the failing files are outside the ABC-5 diff: `packages/app/src/web/*` and `packages/app/src/docker-git/menu-create-shared.ts`. +- Local clone-cache e2e is environment-blocked by remote `DOCKER_HOST` controller discovery; the GitHub `E2E (Clone cache)` job is the authoritative remote e2e signal for this scenario. diff --git a/.kanban/changes/ABC-5/proposal.md b/.kanban/changes/ABC-5/proposal.md index fff2cc17..8256745a 100644 --- a/.kanban/changes/ABC-5/proposal.md +++ b/.kanban/changes/ABC-5/proposal.md @@ -46,7 +46,12 @@ GitHub issue #138: ## Archive Status - Status: implemented -- Spec commit: `f961baa docs(spec): add ABC-5 clone cache plan` -- Implementation commit: `a47e794 feat(clone): reuse repository cache for branch pulls` +- Branch: `vk/d6b8-abc-5-github-138` +- Pull request: https://github.com/ProverCoderAI/docker-git/pull/343 +- Implementation archive head before final audit refresh: `67ed16ec6b543492c80e1e5041bdbf55a934c81c` +- Spec commit after rebase: `7ed9450 docs(spec): add ABC-5 clone cache plan` +- Implementation commit after rebase: `55ffaa3 feat(clone): reuse repository cache for branch pulls` +- Archive baseline commit after rebase: `67ed16e docs(spec): archive ABC-5 clone cache work` - Final behavior: warm-cache clones use the refreshed bare mirror as the local clone source, then restore the authenticated origin and run branch-aware `git pull --ff-only` for normal branch/default branch flows. - E2E note: `bun run e2e:clone-cache` is updated but could not run in the local review environment because the host CLI cannot auto-discover the controller when `DOCKER_HOST=tcp://host.docker.internal:2375` and `DOCKER_GIT_API_URL` is unset. +- Remote CI note: GitHub `E2E (Clone cache)` was still pending at archive time; other remote build/type/test/e2e checks had passed except the repository-wide `Lint` job, which failed on unrelated pre-existing `max-lines` violations outside ABC-5 files. diff --git a/.kanban/changes/ABC-5/tasks.md b/.kanban/changes/ABC-5/tasks.md index 5a1299af..80af8184 100644 --- a/.kanban/changes/ABC-5/tasks.md +++ b/.kanban/changes/ABC-5/tasks.md @@ -40,18 +40,24 @@ ## Archive Verification -- [x] `bun run --filter @effect-template/lib test -- tests/core/templates.test.ts` (48 tests passed) -- [x] `bun run --filter @effect-template/lib test` (51 files, 268 tests passed) -- [x] `bun run --filter @prover-coder-ai/docker-git test` (73 files, 463 tests passed) +- [x] `bun run --filter @effect-template/lib test -- tests/core/templates.test.ts` (49 tests passed) +- [x] `bun run --filter @effect-template/lib test` (52 files, 271 tests passed) +- [x] `bun run --filter @prover-coder-ai/docker-git test` (77 files, 476 tests passed) - [x] `bun run typecheck` -- [x] `bun run lint` +- [x] Scoped lint for `packages/app/src/lib/core/templates-entrypoint/tasks.ts` (0 errors) +- [x] Scoped lint for `packages/lib/src/core/templates-entrypoint/tasks.ts` (0 errors) +- [ ] `bun run lint` failed on unrelated existing `max-lines` and `max-lines-per-function` violations outside ABC-5 changed files. - [x] `bun run build` - [x] Local git repro for warm-cache branch path: bare mirror refresh, clone from mirror, restore origin, `git pull --ff-only origin `. - [ ] `bun run e2e:clone-cache` blocked by local Docker/controller discovery: `DOCKER_HOST=tcp://host.docker.internal:2375` with no reachable `DOCKER_GIT_API_URL`. +- [ ] GitHub `E2E (Clone cache)` pending at archive time. ## Archive Result - Completed branch: `vk/d6b8-abc-5-github-138` -- Spec commit: `f961baa docs(spec): add ABC-5 clone cache plan` -- Implementation commit: `a47e794 feat(clone): reuse repository cache for branch pulls` +- Pull request: https://github.com/ProverCoderAI/docker-git/pull/343 +- Implementation/archive baseline head: `67ed16ec6b543492c80e1e5041bdbf55a934c81c` +- Spec commit: `7ed9450 docs(spec): add ABC-5 clone cache plan` +- Implementation commit: `55ffaa3 feat(clone): reuse repository cache for branch pulls` +- Archive baseline commit: `67ed16e docs(spec): archive ABC-5 clone cache work` - Merge status: not merged to `main`.