feat: configure repository clone cache for #138 #343
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| # ABC-5 Design: Repository Download Cache | ||
|
|
||
| ## Architecture Boundary | ||
|
|
||
| The change belongs in the generated container entrypoint clone shell, rendered by pure TypeScript template functions: | ||
|
|
||
| - Source template: `packages/lib/src/core/templates-entrypoint/tasks.ts` | ||
| - App mirror: `packages/app/src/lib/core/templates-entrypoint/tasks.ts` | ||
| - Compose cache mount: `packages/lib/src/core/templates/docker-compose.ts` and app mirror | ||
|
|
||
| The TypeScript layer remains pure template generation. Git, filesystem, and network operations remain in the generated shell inside the project container. | ||
|
|
||
| ## Proposed Flow | ||
|
|
||
| 1. Resolve `AUTH_REPO_URL` using the existing auth-label logic. | ||
| 2. Resolve deterministic cache key from canonical `REPO_URL`. | ||
| 3. Ensure cache root exists under `/home/<sshUser>/.docker-git/.cache`. | ||
| 4. If a valid cache exists, refresh it before workspace preparation. | ||
| 5. Prepare `TARGET_DIR` from cached data. | ||
| 6. Checkout or create the branch requested by `REPO_REF`. | ||
| 7. Set remotes using the existing fork/upstream logic. | ||
| 8. Mark clone completion with the existing `/run/docker-git/clone.done` or `/run/docker-git/clone.failed` markers. | ||
|
|
||
| ## Cache Shape | ||
|
|
||
| Use the existing bare mirror cache unless implementation proves a working-copy cache is required for the issue semantics. If a working-copy cache is introduced, keep it in a distinct path such as: | ||
|
|
||
| - `/home/<sshUser>/.docker-git/.cache/git-worktrees/<repo-cache-key>` | ||
|
|
||
| The bare mirror path remains: | ||
|
|
||
| - `/home/<sshUser>/.docker-git/.cache/git-mirrors/<repo-cache-key>.git` | ||
|
|
||
| ## Invariants | ||
|
|
||
| - `forall repoUrl: cacheKey(repoUrl)` is deterministic and does not include tokens. | ||
| - `forall clone: cache refresh happens before target workspace checkout when a cache path exists`. | ||
| - `forall target: clone_ok(target) -> target/.git exists`. | ||
| - `forall target, repoRef: clone_ok(target, repoRef) -> target HEAD is repoRef-compatible`. | ||
| - `forall refreshedRefs: refreshedRefs subset refs/heads/* union refs/tags/*` for broad mirror refresh. | ||
| - `forall cache paths: cache path is under ~/.docker-git/.cache and ignored by state repo`. | ||
| - Authenticated URLs may be used for network commands, but tokenized URLs must not become cache keys or persisted remotes. | ||
|
|
||
| ## Branch Handling | ||
|
|
||
| - Plain branch refs: checkout the requested branch from refreshed cache/remote. | ||
| - Issue refs such as `issue-138`: clone default branch, then create or reset local issue branch as existing behavior does. | ||
| - GitHub PR refs `refs/pull/<n>/head`: fetch the specific PR ref into deterministic local branch `pr-<n>`. | ||
| - GitLab MR refs `refs/merge-requests/<n>/head`: fetch the specific MR ref into deterministic local branch `mr-<n>`. | ||
| - Empty `REPO_REF`: use the remote default branch. | ||
|
|
||
| ## Failure Behavior | ||
|
|
||
| - Invalid cache paths are removed or ignored and the flow falls back to network clone. | ||
| - Cache refresh failure should not corrupt an existing valid cache. | ||
| - If final workspace checkout fails, set `CLONE_OK=0` and write the existing failure marker. | ||
|
|
||
| ## Risks | ||
|
|
||
| - Concurrent clones of the same repository may race while refreshing or creating cache paths. | ||
| - Working-copy caches need locking if added. | ||
| - `git pull` semantics differ for branch refs, issue branches, and PR/MR refs; implementation should prefer explicit `fetch` plus checkout/reset where that is more deterministic. | ||
|
|
||
| ## Recommended Implementation Constraint | ||
|
|
||
| Prefer minimal changes to the existing bare mirror flow first. Treat "git pull под нужную нам ветку" as the externally visible invariant: cache is refreshed, then workspace is checked out to the requested branch. Add a working-copy cache only if e2e verification shows the bare mirror flow cannot satisfy the requirement. | ||
|
|
||
| ## Final Design Decision | ||
|
|
||
| The implementation kept the existing bare mirror cache and did not introduce a separate working-copy cache. This preserved the established cache root and state-repo ignore contract while satisfying the branch-refresh invariant: | ||
|
|
||
| 1. Refresh `/home/<sshUser>/.docker-git/.cache/git-mirrors/<repo-cache-key>.git` with branch/tag-only refspecs. | ||
| 2. Clone warm-cache workspaces from that local mirror. | ||
| 3. Restore `origin` to the authenticated repository URL before network fetch/pull. | ||
| 4. Run `git pull --ff-only origin <repoRef>` for normal branch refs, or `git pull --ff-only` for no-ref default-branch clones. | ||
| 5. Keep PR/MR refs on explicit fetch into deterministic local branches. | ||
| 6. Keep issue refs as deterministic local branches after fallback clone. | ||
|
|
||
| No new cache directory was added, so the existing `.cache/git-mirrors/` ignore and untrack behavior remains sufficient. | ||
|
|
||
| ## Archive Invariants | ||
|
|
||
| - Cache key remains derived from canonical `REPO_URL`, not `AUTH_REPO_URL`. | ||
| - Broad cache refresh remains limited to `refs/heads/*` and `refs/tags/*`. | ||
| - Tokenized auth URLs are used for network operations only; final remote normalization still runs through the existing fork/upstream remote block. | ||
| - Clone completion/failure markers remain `/run/docker-git/clone.done` and `/run/docker-git/clone.failed`. | ||
| - CLI/API contracts are unchanged. | ||
|
|
||
| ## Archive Verification Notes | ||
|
|
||
| - The final implementation was rebased onto `origin/main`; implementation/archive baseline was pushed at `67ed16ec6b543492c80e1e5041bdbf55a934c81c`. | ||
| - Task-scope lint passed for both changed template files. | ||
| - Unit, package test, typecheck, and build verification passed locally. | ||
| - Full repository lint is not treated as an ABC-5 regression because the failing files are outside the ABC-5 diff: `packages/app/src/web/*` and `packages/app/src/docker-git/menu-create-shared.ts`. | ||
| - Local clone-cache e2e is environment-blocked by remote `DOCKER_HOST` controller discovery; the GitHub `E2E (Clone cache)` job is the authoritative remote e2e signal for this scenario. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| # ABC-5 Proposal: Repository Download Cache | ||
|
|
||
| ## Summary | ||
|
|
||
| Implement repository download caching for `docker-git clone` so repeated clones of the same repository reuse shared cached git data and refresh that cache before preparing the requested workspace branch. | ||
|
|
||
| ## Requirement | ||
|
|
||
| GitHub issue #138: | ||
|
|
||
| > Что бы один и тот же репозиторий лежал бы в кеше и мы бы грузили данные из кеша + git pull под нужную нам ветку | ||
|
|
||
| ## Current State | ||
|
|
||
| - The clone flow is generated as shell from `packages/lib/src/core/templates-entrypoint/tasks.ts` and mirrored in `packages/app/src/lib/core/templates-entrypoint/tasks.ts`. | ||
| - A shared Docker volume already mounts `/home/dev/.docker-git/.cache` into project containers. | ||
| - Existing cache behavior stores a per-`REPO_URL` bare mirror under `/home/dev/.docker-git/.cache/git-mirrors`. | ||
| - Existing clone behavior uses `--reference-if-able` and `--dissociate`, then performs branch or PR/MR checkout logic in the target workspace. | ||
| - Existing behavior refreshes the bare mirror with `git fetch --prune`, but it does not provide a cached working repository that is updated via branch-aware pull before workspace preparation. | ||
|
|
||
| ## Scope | ||
|
|
||
| - Add or refine clone-cache behavior inside the generated entrypoint clone task. | ||
| - Keep cache state inside the shared cache volume. | ||
| - Preserve existing parsing of `REPO_URL`, `REPO_REF`, issue URLs, PR refs, GitLab merge request refs, auth labels, and fork remotes. | ||
| - Keep state repository sync behavior separate from repository download caching. | ||
| - Extend tests around template rendering and clone-cache e2e behavior. | ||
|
|
||
| ## Non-Goals | ||
|
|
||
| - Do not change CLI argument semantics. | ||
| - Do not merge work into `main`. | ||
| - Do not cache secrets or auth material in repository cache paths. | ||
| - Do not commit cache artifacts into the `.docker-git` state repository. | ||
| - Do not broaden mirror refresh to hosted forge PR/MR refs. | ||
|
|
||
| ## Acceptance Criteria | ||
|
|
||
| - Repeated clones of the same `REPO_URL` use the same cache key/path. | ||
| - The cache is refreshed before it is used for a workspace. | ||
| - The resulting workspace is checked out to the requested `REPO_REF` or deterministic local issue/PR/MR branch. | ||
| - Cache paths remain ignored and untracked by state repository sync. | ||
| - Existing clone flows continue to work for normal branches, issue URLs, GitHub PR refs, GitLab MR refs, and no-ref clones. | ||
| - Tests document the cache reuse and branch-refresh contract. | ||
|
|
||
| ## Archive Status | ||
|
|
||
| - Status: implemented | ||
| - Branch: `vk/d6b8-abc-5-github-138` | ||
| - Pull request: https://github.com/ProverCoderAI/docker-git/pull/343 | ||
| - Implementation archive head before final audit refresh: `67ed16ec6b543492c80e1e5041bdbf55a934c81c` | ||
| - Spec commit after rebase: `7ed9450 docs(spec): add ABC-5 clone cache plan` | ||
| - Implementation commit after rebase: `55ffaa3 feat(clone): reuse repository cache for branch pulls` | ||
| - Archive baseline commit after rebase: `67ed16e docs(spec): archive ABC-5 clone cache work` | ||
| - Final behavior: warm-cache clones use the refreshed bare mirror as the local clone source, then restore the authenticated origin and run branch-aware `git pull --ff-only` for normal branch/default branch flows. | ||
| - E2E note: `bun run e2e:clone-cache` is updated but could not run in the local review environment because the host CLI cannot auto-discover the controller when `DOCKER_HOST=tcp://host.docker.internal:2375` and `DOCKER_GIT_API_URL` is unset. | ||
| - Remote CI note: GitHub `E2E (Clone cache)` was still pending at archive time; other remote build/type/test/e2e checks had passed except the repository-wide `Lint` job, which failed on unrelated pre-existing `max-lines` violations outside ABC-5 files. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| # ABC-5 Tasks | ||
|
|
||
| ## Implementation | ||
|
|
||
| - [x] Update `packages/lib/src/core/templates-entrypoint/tasks.ts` clone-cache flow. | ||
| - [x] Apply the same template update to `packages/app/src/lib/core/templates-entrypoint/tasks.ts`. | ||
| - [x] Keep cache roots under `/home/<sshUser>/.docker-git/.cache`. | ||
| - [x] Preserve existing auth-label and token handling. | ||
| - [x] Preserve existing fork remote behavior after clone. | ||
| - [x] Preserve existing clone completion markers. | ||
| - [x] Ensure cache paths are ignored by state repo sync if new cache directories are added. | ||
|
|
||
| ## Tests | ||
|
|
||
| - [x] Add or extend template tests for cache initialization, refresh, and reuse markers. | ||
| - [x] Add or extend template tests for branch, issue branch, GitHub PR ref, and GitLab MR ref behavior. | ||
| - [x] Extend `scripts/e2e/clone-cache.sh` to verify same repository cache reuse across two clones. | ||
| - [x] Verify second clone uses cache and ends on the requested branch. | ||
| - [x] Verify cache artifacts are not tracked by the state repository. | ||
|
|
||
| ## Verification Commands | ||
|
|
||
| - [x] `bun run --filter @effect-template/lib test` | ||
| - [x] `bun run --filter @prover-coder-ai/docker-git test` | ||
| - [x] `bun run typecheck` | ||
| - [ ] `bun run e2e:clone-cache` (blocked locally: docker-git host CLI cannot auto-discover controller over remote `DOCKER_HOST=tcp://host.docker.internal:2375`) | ||
|
|
||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| ## Out of Scope | ||
|
|
||
| - [ ] No CLI flag changes. | ||
| - [ ] No API contract changes. | ||
| - [ ] No state repo auto-pull changes. | ||
| - [ ] No merge to `main`. | ||
|
|
||
| ## Open Questions | ||
|
|
||
| - Resolved: use the existing bare mirror cache; no separate working-copy cache was added. | ||
| - Deferred: concurrent clone cache refreshes still do not use a lock file. | ||
| - Preserved: failed cache refresh remains a warning/fallback behavior rather than a hard clone failure. | ||
|
|
||
| ## Archive Verification | ||
|
|
||
| - [x] `bun run --filter @effect-template/lib test -- tests/core/templates.test.ts` (49 tests passed) | ||
| - [x] `bun run --filter @effect-template/lib test` (52 files, 271 tests passed) | ||
| - [x] `bun run --filter @prover-coder-ai/docker-git test` (77 files, 476 tests passed) | ||
| - [x] `bun run typecheck` | ||
| - [x] Scoped lint for `packages/app/src/lib/core/templates-entrypoint/tasks.ts` (0 errors) | ||
| - [x] Scoped lint for `packages/lib/src/core/templates-entrypoint/tasks.ts` (0 errors) | ||
| - [ ] `bun run lint` failed on unrelated existing `max-lines` and `max-lines-per-function` violations outside ABC-5 changed files. | ||
| - [x] `bun run build` | ||
| - [x] Local git repro for warm-cache branch path: bare mirror refresh, clone from mirror, restore origin, `git pull --ff-only origin <branch>`. | ||
| - [ ] `bun run e2e:clone-cache` blocked by local Docker/controller discovery: `DOCKER_HOST=tcp://host.docker.internal:2375` with no reachable `DOCKER_GIT_API_URL`. | ||
| - [ ] GitHub `E2E (Clone cache)` pending at archive time. | ||
|
There was a problem hiding this comment. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Проверяем текущий статус E2E check для PR `#343`
gh pr view 343 --json statusCheckRollup --jq '.statusCheckRollup[] | select(.context | contains("E2E") or contains("Clone cache")) | {context: .context, state: .state, conclusion: .conclusion}'Repository: ProverCoderAI/docker-git Length of output: 111 🏁 Script executed: #!/bin/bash
set -euo pipefail
# 1) Печатаем тип/наличие statusCheckRollup
gh pr view 343 --json statusCheckRollup --jq '{
statusCheckRollup_type: (.statusCheckRollup | type),
statusCheckRollup_is_null: (.statusCheckRollup == null)
}'
# 2) Достаём только контексты, где context не null и содержит E2E/Clone cache
gh pr view 343 --json statusCheckRollup --jq '
(.statusCheckRollup // [])
| map(select((.context? // "") | test("E2E|Clone cache")))
| map({context: .context, state: .state, conclusion: .conclusion})
'Repository: ProverCoderAI/docker-git Length of output: 140 🏁 Script executed: #!/bin/bash
set -euo pipefail
gh pr view 343 --json statusCheckRollup --jq '
(.statusCheckRollup // [])
| map({context: .context, state: .state, conclusion: .conclusion})
| sort_by(.context)
| .[0:200]
'Repository: ProverCoderAI/docker-git Length of output: 1071 🏁 Script executed: #!/bin/bash
set -euo pipefail
# Покажем структуру одного элемента statusCheckRollup (первые 1-3 штуки)
gh pr view 343 --json statusCheckRollup --jq '
(.statusCheckRollup // [])
| [.[0], .[1], .[2]]
'Repository: ProverCoderAI/docker-git Length of output: 964 🏁 Script executed: #!/bin/bash
set -euo pipefail
gh pr view 343 --json statusCheckRollup --jq '
(.statusCheckRollup // [])
| map({
name: .name,
workflowName: .workflowName,
status: .status,
conclusion: .conclusion,
detailsUrl: .detailsUrl
})
| sort_by(.name)
'Repository: ProverCoderAI/docker-git Length of output: 3700 Подождите завершения всех E2E check’ов перед merge (Clone cache уже SUCCESS)
🤖 Prompt for AI Agents |
||
|
|
||
| ## Archive Result | ||
|
|
||
| - Completed branch: `vk/d6b8-abc-5-github-138` | ||
| - Pull request: https://github.com/ProverCoderAI/docker-git/pull/343 | ||
| - Implementation/archive baseline head: `67ed16ec6b543492c80e1e5041bdbf55a934c81c` | ||
| - Spec commit: `7ed9450 docs(spec): add ABC-5 clone cache plan` | ||
| - Implementation commit: `55ffaa3 feat(clone): reuse repository cache for branch pulls` | ||
| - Archive baseline commit: `67ed16e docs(spec): archive ABC-5 clone cache work` | ||
| - Merge status: not merged to `main`. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
Repository: ProverCoderAI/docker-git
Length of output: 2788
Блокировка merge: E2E ещё не завершён (PR
#343)В
.kanban/changes/ABC-5/proposal.md(стр. 57) сказано, что E2E (Clone cache) был “still pending at archive time”, и сейчас по PR#343E2E проверки всё ещё в состоянииpending— в том числе:E2E (Clone cache),E2E (OpenCode),E2E (Login context),E2E (Runtime volumes + SSH),E2E (Clone auto-open SSH). Merge стоит делать только после успешного завершения всех E2E.🤖 Prompt for AI Agents