docs(codebuilder-e2e): plan for reusable Code Builder e2e toolkit - W-23385030 - #7877
Draft
kylewalke wants to merge 29 commits into
Draft
docs(codebuilder-e2e): plan for reusable Code Builder e2e toolkit - W-23385030#7877kylewalke wants to merge 29 commits into
kylewalke wants to merge 29 commits into
Conversation
…385030 Add a container e2e harness that drives the desktop extension build over a browser at the real code-builder-images code-server runtime, swapping freshly -built (unreleased) monorepo VSIX in at runtime so specs catch bugs before marketplace publish. - playwright-vscode-ext: createContainerConfig + createContainerTest - core: containerFixtures, seed container spec, config, test:container script - workflow codeBuilderE2E.yml (dispatch/call) + swap/verify scripts - ADR 0022 and doc updates
…85030 The App ID is a public identifier, not a credential — move it to an org variable (vars.CB_GHCR_APP_ID). Only the App private key stays a secret.
…W-23385030 Match the desktop e2e leaves: test the artifacts about to ship, not a rebuild. The Code Builder workflow now takes runId + artifactName inputs and downloads the VSIX from an upstream Build All run, then swaps those into the container. Version gate asserts the shipping version. Docs/ADR updated to match.
…- W-23385030 The workbench-poll loops sleep 2s per iteration but logged the iteration counter as seconds. Multiply by 2 so "up after Ns" is real elapsed time.
…e workflow Revert before merge.
…sion registration
…prefix collisions
…out of Restricted Mode
…5030 Port the three Code Builder e2e helpers from bash to ts-node scripts in scripts/, matching the repo's script convention. Behavior is unchanged: same pull -> swap -> restart -> version-gate -> run pipeline. - scripts/codeBuilderLocalE2E.ts one-command local loop (test:container:local) - scripts/codeBuilderSwapExtensions.ts runtime extension swap - scripts/codeBuilderVerifyExtensions.ts version gate The workflow swap/gate steps and the test:container:local npm script now invoke the .ts files via ts-node. Uses execFileSync with arg arrays instead of shell interpolation, removing the shell-injection surface on the docker/gh/sf calls.
The local Code Builder e2e loop no longer leans on each dev's gh credential (with a read:packages scope refresh) to pull the private CB image. It now fetches a shared SVC_IDEE bot pull token from 1Password via the op CLI, so there's no per-dev PAT to manage and no browser scope-refresh flow. - Preflight requires op (1Password CLI); collects it alongside docker in the consolidated missing-prereqs report with a copy-paste install command. - Token resolves at runtime from OP_GHCR_ITEM (SVC_IDE_BOT_GHCR_READ_TOKEN in the "Platform Dev Tools Team" vault), never landing in the repo. Overridable via OP_GHCR_ITEM / OP_ACCOUNT. - CR_PAT stays as the escape hatch for devs who prefer their own classic PAT. - gh is now only needed for the --run-id CI-artifact path, not the image pull. CI is unchanged — it still pulls with the repo GITHUB_TOKEN.
…- W-23385030 The extension VSIX version isn't release-bumped, so it matches the marketplace build too — the semver alone can't tell a reader whether the assets under test are shipping bytes or the unreleased pre-release build. CI: the "List VSIX under test" step becomes "Provenance of VSIX under test", resolving the source Build All runId via `gh run view` and printing a banner (workflow, trigger, branch, commit + title, build timestamp, run URL) before the file listing. Local loop: --run-id logs the same upstream-run provenance; a working-tree build logs the current git branch + HEAD (flagging a dirty tree). Both note the semver is not release-bumped.
…ns - W-23385030 The runtime extension swap and the version gate both lean on extension semver, which is fragile in two ways: the swap rides code-builder-images' strictly-newer relink rule (a contract that team owns), and the gate compares installed semver against the artifact semver rather than bytes — so an unreleased build sharing a version with the baked marketplace copy can false-green if the swap silently no-ops. Records the unconditional-wipe option (drops the semver-precedence dependency) and the residual content-identity gap (needs a content hash or build stamp) as Considered Options / Consequences. No mechanism change in this PR.
…W-23385030 The workbench opens whatever the image's sfdx-setup.sh seeds on first boot (SFDX_COBU_PROJECTNAME -> bare generated project; the GitHub-clone branch is dormant), gated behind ~/.codebuilder so it never re-runs on our swap restart. Nothing is seeded today, so specs needing real metadata have nothing to drive. Records two paths: create metadata mid-spec via extension commands, or the candidate volume-mount of an in-repo fixture project (writing coder.json via docker exec, decoupled from the image's first-boot SFDX_COBU_* env path). Notes the disable-trust / workbench-wait must track the opened folder. No code change.
…roject - W-23385030
…ub user - W-23385030
# Conflicts: # packages/playwright-vscode-ext/src/utils/nonCriticalErrorPatterns.ts
…23385030 The local build (npm run vscode:package) produces both a modern and a legacy VSIX for core/lwc/lightning/apex-debugger/apex-oas. The collection loop grabbed every *.vsix per package dir, so the swap unpacked two override dirs per extension and the verify gate failed with 'expected exactly 1 override dir, found 2'. Collect only each package's own-version (modern) VSIX.
…g display - W-23385030 Recent sf CLI versions redact accessToken in 'sf org display --json', returning a '[REDACTED] Use sf org auth show-access-token ...' placeholder instead of the real token. Both the local script and the CI workflow passed that placeholder as SF_ACCESS_TOKEN into the container, so the container's start-time org login failed (exit_code=1), no config was set, and the Config List spec saw 'No results found'. Source the token from 'sf org auth show-access-token --json' in both places. Verified locally: container auto-login succeeds, all container specs pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A planning document for turning the proven Code Builder e2e pipeline from #7718 into a reusable, incrementally-delivered TS package (
@salesforce/code-builder-e2e).docs/plans/code-builder-e2e-toolkit.mdWhy
#7718 proved the pipeline (pull → swap → restart → gate → run) but it lives as repo-specific
ts-nodescripts, and it carries the false-green bug ADR 0022 documents: the gate compares installed semver, not bytes, so a silently no-op'd swap passes green. This plan turns the scripts into a layered package and closes that bug via the ADR's recommended-but-not-yet-adopted direction: unconditional wipe by publisher glob + content check.Key decisions captured
package.json+mainbundle), not just semver.ContainerHandleand an Effect-SchemaManifest(swap emits → verify consumes → persisted as provenance).sf-awareresolveOrgBootEnv(carries theshow-access-tokenredaction lesson).Note
Doc only — no code changes. Separated from the #7846 VSIX-dedupe fix branch on purpose for independent review.