Orchestrate a build with the CNB lifecycle Not Working

[wilsonfv]

Summary

The tutorial Orchestrate a build with the CNB lifecycle does not work because at the Analyze phase, somehow it had fetched an invalid run-image image reference SHA number from local docker daemon so at the Export phase, it couldn't produce an image since run-image image reference SHA number did not exist

Environment

• https://github.com/buildpacks/lifecycle/tree/65776dffb9353b287e62ef8745e1954f84441ce9
• https://github.com/buildpacks/samples/tree/fa282af819c4b21f4c4d50d27486e21f70f55a68
• Apple M4
• Docker Desktop 4.60.1 (218372)
• Docker Daemon

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 29.2.0
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: dea7da592f5d1d2b7755e3a161be07f43fad8f75
 runc version: v1.3.4-0-gd6d73eb8
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.12.67-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 10
 Total Memory: 7.653GiB
 Name: docker-desktop
 ID: dcd8ea0e-2705-4c03-a378-074d243c74dc
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/wilsonfv/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false

Reproduction

docker inspect

docker inspect on image arm64v8/ubuntu:latest already pulled onto local daemon
Notice the base image SHA number is 2953d1b52d8f0d14dca537cea7bc939119d9c9c6c0e57357897bcc89ca6c4e44

$ docker image ls
                                                                                          i Info →   U  In Use
IMAGE                   ID             DISK USAGE   CONTENT SIZE   EXTRA
arm64v8/ubuntu:latest   2953d1b52d8f        141MB         30.8MB        

$ docker inspect arm64v8/ubuntu:latest 
[
    {
        "Id": "sha256:2953d1b52d8f0d14dca537cea7bc939119d9c9c6c0e57357897bcc89ca6c4e44",
        "RepoTags": [
            "arm64v8/ubuntu:latest"
        ],
        "RepoDigests": [
            "arm64v8/ubuntu@sha256:2953d1b52d8f0d14dca537cea7bc939119d9c9c6c0e57357897bcc89ca6c4e44"
        ],
        "Created": "2026-01-13T05:40:17.823477488Z",
        "Config": {
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/bin/bash"
            ],
            "Labels": {
                "org.opencontainers.image.ref.name": "ubuntu",
                "org.opencontainers.image.version": "24.04"
            }
        },
        "Architecture": "arm64",
        "Variant": "v8",
        "Os": "linux",
        "Size": 28868043,
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:af7f55edf6d6953f845c9ccd22ecfb679043d792775d2bf4c210ff22485a7397"
            ]
        },
        "Metadata": {
            "LastTagTime": "2026-02-15T13:50:46.207482213Z"
        },
        "Descriptor": {
            "mediaType": "application/vnd.oci.image.index.v1+json",
            "digest": "sha256:2953d1b52d8f0d14dca537cea7bc939119d9c9c6c0e57357897bcc89ca6c4e44",
            "size": 1482
        },
        "Identity": {
            "Pull": [
                {
                    "Repository": "docker.io/arm64v8/ubuntu"
                }
            ]
        }
    }
]

Analyze

After Analyze step, it had fetched an invalid and non-existent image SHA number 0406aba87cff4e77d400a3e9e85acf0e07cedf49d633b55d89928bdfb8bfa0fd

$ ${CNB_LIFECYCLE_PATH}/analyzer -log-level debug -daemon -layers="./layers" -run-image arm64v8/ubuntu:latest apps/bash-script

Starting analyzer...
Parsing inputs...
Ensuring privileges...
Executing command...
Docker did not find image apps/bash-script with platform linux/arm64; retrying without specifying a platform
Timer: Analyzer started at 2026-02-18T13:31:19+08:00
Image with name "apps/bash-script" not found
Found image with identifier "0406aba87cff4e77d400a3e9e85acf0e07cedf49d633b55d89928bdfb8bfa0fd"
Timer: Analyzer ran for 37.167µs and ended at 2026-02-18T13:31:19+08:00
Run image info in analyzed metadata is: 
{"Reference":"0406aba87cff4e77d400a3e9e85acf0e07cedf49d633b55d89928bdfb8bfa0fd","Image":"arm64v8/ubuntu:latest","Extend":false,"target":{"os":"linux","arch":"arm64","arch-variant":"v8"}}

$ cat layers/analyzed.toml 
[image]
  reference = ""

[metadata]
  [metadata.config]
    sha = ""
  [metadata.launcher]
    sha = ""
  [metadata.process-types]
    sha = ""
  [metadata.run-image]
    top-layer = ""
    reference = ""

[run-image]
  reference = "0406aba87cff4e77d400a3e9e85acf0e07cedf49d633b55d89928bdfb8bfa0fd"
  image = "arm64v8/ubuntu:latest"
  [run-image.target]
    os = "linux"
    arch = "arm64"
    arch-variant = "v8"

Detect

$ ${CNB_LIFECYCLE_PATH}/detector -log-level debug -layers="./layers" -order="./order.toml" -buildpacks="./buildpacks" -app apps/bash-script
Starting detector...
Parsing inputs...
Ensuring privileges...
Executing command...
Timer: Detector started at 2026-02-18T13:38:13+08:00
Checking for match against descriptor: {linux amd64  []}
target distro name/version labels not found, reading /etc/os-release file
Checking for match against descriptor: {linux amd64  []}
Checking for match against descriptor: {linux amd64  []}
Checking for match against descriptor: {linux   []}
Checking for match against descriptor: {linux   []}
======== Error: samples/java-maven@0.0.3 ========
unable to satisfy target os/arch constraints; run image: {"os":"linux","arch":"arm64","arch-variant":"v8"}, buildpack: [{"os":"linux","arch":"amd64"}]
======== Results ========
err:  samples/java-maven@0.0.3
======== Error: samples/kotlin-gradle@0.0.3 ========
unable to satisfy target os/arch constraints; run image: {"os":"linux","arch":"arm64","arch-variant":"v8"}, buildpack: [{"os":"linux","arch":"amd64"}]
======== Results ========
err:  samples/kotlin-gradle@0.0.3
======== Error: samples/ruby-bundler@0.0.2 ========
unable to satisfy target os/arch constraints; run image: {"os":"linux","arch":"arm64","arch-variant":"v8"}, buildpack: [{"os":"linux","arch":"amd64"}]
======== Results ========
err:  samples/ruby-bundler@0.0.2
======== Results ========
pass: samples/hello-world@0.0.2
pass: samples/hello-moon@0.0.2
Resolving plan... (try #1)
samples/hello-world 0.0.2
samples/hello-moon  0.0.2
Timer: Detector ran for 925.158959ms and ended at 2026-02-18T13:38:14+08:00

Restore

$ ${CNB_LIFECYCLE_PATH}/restorer -log-level debug -daemon -layers="./layers" -group="./layers/group.toml" -cache-dir="./cache" -analyzed="./layers/analyzed.toml"
Starting restorer...
Parsing inputs...
Ensuring privileges...
Executing command...
No run metadata found at path "/cnb/run.toml"
Run image info in analyzed metadata is: 
{"Reference":"0406aba87cff4e77d400a3e9e85acf0e07cedf49d633b55d89928bdfb8bfa0fd","Image":"arm64v8/ubuntu:latest","Extend":false,"target":{"os":"linux","arch":"arm64","arch-variant":"v8"}}
Timer: Restorer started at 2026-02-18T13:39:08+08:00
Restoring Layer Metadata
Reading buildpack directory: /tmp/tutorial/layers/samples_hello-world
Reading buildpack directory: /tmp/tutorial/layers/samples_hello-moon
Reading Buildpack Layers directory /tmp/tutorial/layers
Reading buildpack directory: /tmp/tutorial/layers/samples_hello-world
Reading Buildpack Layers directory /tmp/tutorial/layers
Reading buildpack directory: /tmp/tutorial/layers/samples_hello-moon
Timer: Restorer ran for 284.584µs and ended at 2026-02-18T13:39:08+08:00

Build

$ ${CNB_LIFECYCLE_PATH}/builder -log-level debug -layers="./layers" -group="./layers/group.toml" -analyzed="./layers/analyzed.toml" -plan="./layers/plan.toml" -buildpacks="./buildpacks" -app="./workspace" -platform="./platform"
Starting builder...
Parsing inputs...
Ensuring privileges...
Executing command...
Timer: Builder started at 2026-02-18T13:40:35+08:00
target distro name/version labels not found, reading /etc/os-release file
Running build for buildpack samples/hello-world@0.0.2
Looking up buildpack
Finding plan
Creating plan directory
Preparing paths
Running build command
---> Hello World buildpack
     platform_dir files:
       total 0
       drwxr-xr-x@ 2 wilsonfv  wheel   64 Feb 18 13:40 .
       drwxr-xr-x@ 9 wilsonfv  wheel  288 Feb 18 13:40 ..
     env_dir: /tmp/tutorial/platform/env
     env vars:
       declare -x CNB_BP_PLAN_PATH="/var/folders/_z/vrm5yb097d33zk81cprgk8000000gn/T/samples_hello-world-926742123/samples_hello-world/plan.toml"
       declare -x CNB_BUILDPACK_DIR="/tmp/tutorial/buildpacks/samples_hello-world/0.0.2"
       declare -x CNB_LAYERS_DIR="/tmp/tutorial/layers/samples_hello-world"
       declare -x CNB_PLATFORM_DIR="/tmp/tutorial/platform"
       declare -x CNB_TARGET_ARCH="arm64"
       declare -x CNB_TARGET_ARCH_VARIANT="v8"
       declare -x CNB_TARGET_OS="linux"
       declare -x HOME="/Users/wilsonfv"
       declare -x OLDPWD
       declare -x PATH="/Users/wilsonfv/go/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/usr/local/go/bin:/opt/homebrew/bin"
       declare -x PWD="/private/tmp/tutorial/workspace"
       declare -x SHLVL="1"
     layers_dir: /tmp/tutorial/layers/samples_hello-world
     plan_path: /var/folders/_z/vrm5yb097d33zk81cprgk8000000gn/T/samples_hello-world-926742123/samples_hello-world/plan.toml
     plan contents:
       [[entries]]
         name = "some-world"
       
       [[entries]]
         name = "some-world"
         [entries.metadata]
           world = "Earth-616"

---> Done
Processing layers
Updating environment
Reading output files
Updating buildpack processes
Updating process list
Finished running build for buildpack samples/hello-world@0.0.2
Running build for buildpack samples/hello-moon@0.0.2
Looking up buildpack
Finding plan
Creating plan directory
Preparing paths
Running build command
---> Hello Moon buildpack
     env_dir: /tmp/tutorial/platform/env
     env vars:
       declare -x CNB_BP_PLAN_PATH="/var/folders/_z/vrm5yb097d33zk81cprgk8000000gn/T/samples_hello-moon-1665263822/samples_hello-moon/plan.toml"
       declare -x CNB_BUILDPACK_DIR="/tmp/tutorial/buildpacks/samples_hello-moon/0.0.2"
       declare -x CNB_LAYERS_DIR="/tmp/tutorial/layers/samples_hello-moon"
       declare -x CNB_PLATFORM_DIR="/tmp/tutorial/platform"
       declare -x CNB_TARGET_ARCH="arm64"
       declare -x CNB_TARGET_ARCH_VARIANT="v8"
       declare -x CNB_TARGET_OS="linux"
       declare -x HOME="/Users/wilsonfv"
       declare -x OLDPWD
       declare -x PATH="/Users/wilsonfv/go/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/usr/local/go/bin:/opt/homebrew/bin"
       declare -x PWD="/private/tmp/tutorial/workspace"
       declare -x SHLVL="1"
     layers_dir: /tmp/tutorial/layers/samples_hello-moon
     plan_path: /var/folders/_z/vrm5yb097d33zk81cprgk8000000gn/T/samples_hello-moon-1665263822/samples_hello-moon/plan.toml
     plan contents:

---> Done
Processing layers
Updating environment
Reading output files
Updating buildpack processes
Updating process list
Finished running build for buildpack samples/hello-moon@0.0.2
Copying SBOM files
Creating SBOM files for legacy BOM
Listing processes
Timer: Builder ran for 1.336122083s and ended at 2026-02-18T13:40:36+08:00

Export

As you can see below, it couldn't fetch run image by SHA number

$ ${CNB_LIFECYCLE_PATH}/exporter --log-level debug -launch-cache "./cache" -daemon -cache-dir "./cache" -analyzed "./layers/analyzed.toml" -group "./layers/group.toml" -layers="./layers" -app "./workspace" -launcher="${CNB_LINUX_LAUNCHER_PATH}" apps/bash-script
Starting exporter...
Parsing inputs...
Ensuring privileges...
Executing command...
No project metadata found at path "./layers/project-metadata.toml", project metadata will not be exported
Docker did not find image 0406aba87cff4e77d400a3e9e85acf0e07cedf49d633b55d89928bdfb8bfa0fd with platform linux/arm64; retrying without specifying a platform
No run metadata found at path "/cnb/run.toml"
Timer: Exporter started at 2026-02-18T13:42:03+08:00
Warning: Did not find SBOM lifecycle.sbom.cdx.json in /cnb/lifecycle
Warning: Did not find SBOM lifecycle.sbom.spdx.json in /cnb/lifecycle
Warning: Did not find SBOM lifecycle.sbom.syft.json in /cnb/lifecycle
Warning: Did not find SBOM launcher.sbom.cdx.json in /cnb/lifecycle
Warning: Did not find SBOM launcher.sbom.spdx.json in /cnb/lifecycle
Warning: Did not find SBOM launcher.sbom.syft.json in /cnb/lifecycle
Timer: Exporter ran for 30.959µs and ended at 2026-02-18T13:42:03+08:00
ERROR: failed to export: get run image top layer SHA: image has no layers

Manual Fix

If I manually update layers/analyzed.toml with correct run image SHA number 2953d1b52d8f0d14dca537cea7bc939119d9c9c6c0e57357897bcc89ca6c4e44 and rerun Export, it will work

$ cat layers/analyzed.toml 
[image]
  reference = ""

[metadata]
  [metadata.config]
    sha = ""
  [metadata.launcher]
    sha = ""
  [metadata.process-types]
    sha = ""
  [metadata.run-image]
    top-layer = ""
    reference = ""

[run-image]
  reference = "2953d1b52d8f0d14dca537cea7bc939119d9c9c6c0e57357897bcc89ca6c4e44"
  image = "arm64v8/ubuntu:latest"
  [run-image.target]
    os = "linux"
    arch = "arm64"
    arch-variant = "v8"

$ ${CNB_LIFECYCLE_PATH}/exporter --log-level debug -launch-cache "./cache" -daemon -cache-dir "./cache" -analyzed "./layers/analyzed.toml" -group "./layers/group.toml" -layers="./layers" -app "./workspace" -launcher="${CNB_LINUX_LAUNCHER_PATH}" apps/bash-script
Starting exporter...
Parsing inputs...
Ensuring privileges...
Executing command...
No project metadata found at path "./layers/project-metadata.toml", project metadata will not be exported
No run metadata found at path "/cnb/run.toml"
Timer: Exporter started at 2026-02-18T13:55:10+08:00
Warning: Did not find SBOM lifecycle.sbom.cdx.json in /cnb/lifecycle
Warning: Did not find SBOM lifecycle.sbom.spdx.json in /cnb/lifecycle
Warning: Did not find SBOM lifecycle.sbom.syft.json in /cnb/lifecycle
Warning: Did not find SBOM launcher.sbom.cdx.json in /cnb/lifecycle
Warning: Did not find SBOM launcher.sbom.spdx.json in /cnb/lifecycle
Warning: Did not find SBOM launcher.sbom.syft.json in /cnb/lifecycle
Reading buildpack directory: /tmp/tutorial/layers/samples_hello-world
Processing buildpack directory: /tmp/tutorial/layers/samples_hello-world
Reading buildpack directory: /tmp/tutorial/layers/samples_hello-moon
Reading buildpack directory item: launch.toml
Processing buildpack directory: /tmp/tutorial/layers/samples_hello-moon
Found SBOM of type launch for at /tmp/tutorial/layers/sbom/launch
Reusing tarball for layer "buildpacksio/lifecycle:launch.sbom" with SHA: sha256:76d9fd4935578553e1d1c0d1b08c2229850891f54b384b32d16090d0cb5f15f3
Adding layer 'buildpacksio/lifecycle:launch.sbom'
Layer 'buildpacksio/lifecycle:launch.sbom' SHA: sha256:76d9fd4935578553e1d1c0d1b08c2229850891f54b384b32d16090d0cb5f15f3
Adding app layers
Layer 'slice-1' SHA: sha256:928b22c2cbd5762cf07786d628a22cedd5933b63a79e526a798ca67229d07d02
Added 1/1 app layer(s)
Reusing tarball for layer "buildpacksio/lifecycle:launcher" with SHA: sha256:4bd8ecd3d22f51ad0a2596c1dbbf33a316a617a61240fe6bfb954ef61421462a
Adding layer 'buildpacksio/lifecycle:launcher'
Layer 'buildpacksio/lifecycle:launcher' SHA: sha256:4bd8ecd3d22f51ad0a2596c1dbbf33a316a617a61240fe6bfb954ef61421462a
Reusing tarball for layer "buildpacksio/lifecycle:config" with SHA: sha256:6b9ee88e7f2acdb40092aceaa8b3ff47774e6330e71fb512156228957f2626cc
Adding layer 'buildpacksio/lifecycle:config'
Layer 'buildpacksio/lifecycle:config' SHA: sha256:6b9ee88e7f2acdb40092aceaa8b3ff47774e6330e71fb512156228957f2626cc
Reusing tarball for layer "buildpacksio/lifecycle:process-types" with SHA: sha256:e598f2ee7e8b980b4a96ba60481a083f241c970d4c6960ce0f4a200e985f996f
Adding layer 'buildpacksio/lifecycle:process-types'
Layer 'buildpacksio/lifecycle:process-types' SHA: sha256:e598f2ee7e8b980b4a96ba60481a083f241c970d4c6960ce0f4a200e985f996f
Adding label 'io.buildpacks.lifecycle.metadata'
Adding label 'io.buildpacks.build.metadata'
Adding label 'io.buildpacks.project.metadata'
Setting CNB_LAYERS_DIR=/tmp/tutorial/layers
Setting CNB_APP_DIR=/tmp/tutorial/workspace
Setting CNB_PLATFORM_API=0.14
Setting CNB_DEPRECATION_MODE=quiet
Prepending /cnb/process and /cnb/lifecycle to PATH
Setting WORKDIR: '/tmp/tutorial/workspace'
no default process type
Setting ENTRYPOINT: '/cnb/lifecycle/launcher'
Timer: Saving apps/bash-script... started at 2026-02-18T13:55:10+08:00
Saving apps/bash-script...
*** Images (c68422e8a00a):
      apps/bash-script

*** Image ID: c68422e8a00ad065d18b87f753a3242db6d17cc2139de70310d8b5b16e36f04c

*** Manifest Size: 1158
Timer: Saving apps/bash-script... ran for 2.645034958s and ended at 2026-02-18T13:55:13+08:00
Timer: Exporter ran for 2.679033666s and ended at 2026-02-18T13:55:13+08:00
Timer: Cache started at 2026-02-18T13:55:13+08:00
Reading buildpack directory: /tmp/tutorial/layers/samples_hello-world
Reading buildpack directory: /tmp/tutorial/layers/samples_hello-moon
Reading buildpack directory item: launch.toml
Timer: Cache ran for 122.792µs and ended at 2026-02-18T13:55:13+08:00
Warning: Failed to export cache: setting cache metadata: creating metadata file '/tmp/tutorial/cache/staging/io.buildpacks.lifecycle.cache.metadata': open /tmp/tutorial/cache/staging/io.buildpacks.lifecycle.cache.metadata: no such file or directory

[AidanDelaney]

This is an amazing bug report. Thank you for the detail.

[AidanDelaney]

The lifecycle's analyzer phase resolves the wrong image identifier for the run image when operating in daemon mode on an ARM64 machine with Docker Desktop's containerd-based image store.

What's happening step by step:

1. The user pulls arm64v8/ubuntu:latest. Docker stores it with image ID 2953d1b52d8f.... This is the image index (manifest list) digest, as confirmed by the docker inspect output showing media type application/vnd.oci.image.index.v1+json.

2. When the analyzer runs with -daemon -run-image arm64v8/ubuntu:latest, it resolves the run image reference to a different SHA: 0406aba87cff.... This SHA is most likely the digest of the platform-specific manifest (the arm64/v8 manifest within the multi-arch image index) — not the top-level image index ID that Docker uses to identify the image locally.

3. This 0406aba... SHA gets written into layers/analyzed.toml as the run image reference.

4. When the exporter later tries to load the run image using that reference (0406aba...), Docker can't find it — because Docker's daemon indexes images by the image index ID (2953d1b...), not by individual platform manifest digests. The exporter fails with: ERROR: failed to export: get run image top layer SHA: image has no layers.

Root cause: The lifecycle's daemon image-resolution code is returning a platform-specific manifest digest from within a multi-arch image index, rather than the image ID that the Docker daemon actually uses to identify the pulled image. This is likely a regression or incompatibility with Docker Desktop's newer containerd-based image store, which handles multi-arch images differently than the older Docker image store.

I'll report this in lifecycle.