Skip to content

feat(kubernetes): support direct Deployment images - #1098

Open
microagi-andy wants to merge 1 commit into
alchemy-run:mainfrom
MicroAGI-Labs:codex/kubernetes-deployment-direct-images
Open

feat(kubernetes): support direct Deployment images#1098
microagi-andy wants to merge 1 commit into
alchemy-run:mainfrom
MicroAGI-Labs:codex/kubernetes-deployment-direct-images

Conversation

@microagi-andy

Copy link
Copy Markdown
Contributor

Summary

  • add an opt-in imageStrategy: "direct" mode for pre-built Kubernetes.Deployment images
  • preserve the current managed-registry mirroring behavior as the default
  • keep workload identity and synthesized Kubernetes resources unchanged while bypassing local Docker/ECR for direct images
  • wait for the new Deployment rollout to complete before deleting registry state from a previous mirrored deployment
  • mark registry unstable during direct/mirror transitions so plans do not retain stale attributes

Why

Some EKS workloads pull from registries that already provide their own authentication and caching path. Mirroring those images through ECR forces the deploy machine to pull/tag/push through local Docker and prevents a transparent in-cluster registry cache from owning the data path.

This keeps mirroring backward-compatible while allowing an explicit direct reference:

yield* Kubernetes.Deployment("Cache", {
  cluster,
  image: "registry.example.com/cache@sha256:...",
  imageStrategy: "direct",
});

On a mirrored-to-direct transition, cleanup happens only after Kubernetes reports the rollout complete (new replicas updated and available, no old replicas remaining).

Verification

  • bun tsc --noEmit -p packages/alchemy/tsconfig.json --pretty false
  • bun run test test/Kubernetes/Deployment.test.ts (5 passed, 2 slow E2E tests skipped)

@sam-goodwin

Copy link
Copy Markdown
Contributor

I wonder if we can use Docker.RemoteImage instead of imageStrategy

see: https://alchemy.run/providers/docker/remoteimage/

@microagi-andy
microagi-andy force-pushed the codex/kubernetes-deployment-direct-images branch from 0820c74 to 60860f5 Compare August 5, 2026 08:46
Comment on lines +51 to +54
Effect.provide([NodeFileSystem.layer, Path.layer]),
Effect.provideService(Stack, unitStack),
Effect.provideService(Stage, unitStack.stage),
Effect.provideService(InstanceId, "0123456789abcdef0123456789abcdef"),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build and provide a single Layer, don't use subsequent Effect.provide*

Even in tests because otherwise we pollute context

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in c2be7e6: the test now composes NodeFileSystem, Path, Stack, Stage, and InstanceId with one Layer.mergeAll(...) and provides that layer in a single Effect.provide(...).

@microagi-andy
microagi-andy force-pushed the codex/kubernetes-deployment-direct-images branch from 60860f5 to d3f96df Compare August 5, 2026 08:50
@microagi-andy

Copy link
Copy Markdown
Contributor Author

I checked Docker.RemoteImage, but it preserves the behavior this change needs to avoid: its reconcile unconditionally calls docker.image.pull(...) and then docker.image.inspect(...) (and optionally tag/push), so it still requires a Docker context and materializes the image on the deploy host. alwaysPull: false only suppresses subsequent diffs; the create still pulls.

Here the pod is intentionally pulling through an in-cluster AWS layer cache which obtains short-lived GAR credentials via projected EKS OIDC -> GCP WIF. The deploy host neither has nor should need that registry auth or Docker. The missing capability is specifically for Kubernetes.Deployment to treat a prebuilt image reference as an opaque pod-spec value while retaining its ServiceAccount / Pod Identity / Service lifecycle.

I also tightened the latest revision so this option is Deployment-only internally as well as in the public type; untyped Kubernetes.Job props cannot accidentally enter the direct path without Job-specific cleanup semantics.

Happy to adjust the API spelling if you prefer something like mirrorImage: false, but Docker.RemoteImage itself cannot express this no-local-Docker path.

@microagi-andy
microagi-andy force-pushed the codex/kubernetes-deployment-direct-images branch from d3f96df to c2be7e6 Compare August 5, 2026 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants