Add support for OCI artifacts

[onedr0p]

Describe the problem/challenge you have

OCI registries are increasingly used to distribute non-image content — flux push artifact, oras push, helm OCI charts, WASM modules, policy bundles. Pulling these with vendir today means either:

1. Using image: and hoping its filesystem-overlay extraction works against a non-image manifest, or
2. Scripting oras pull / flux pull artifact out-of-band and feeding the result back via directory: — which loses the digest lock and the Renovate-trackable shape that's the whole point of vendir.

Describe the solution you'd like

A first-class ociArtifact: source that pulls via the OCI distribution APIs directly, extracts each layer blob's tar(.gz) into path, and records the resolved digest in vendir.lock.yml. Auth reuses the image: source's docker-config model; cosign verification is opt-in.

- path: vendor/k8s-schemas
  contents:
    - path: .
      ociArtifact:
        image: ghcr.io/home-operations/k8s-schemas
        tag: latest
        mediaTypes:
          - application/vnd.cncf.flux.content.v1.tar+gzip
        cosign:
          certificateOidcIssuer: https://token.actions.githubusercontent.com

Anything else you would like to add:

• Natural implementation dep: oras-go.
• Worth a separate source type rather than extending image: — artifacts have no overlay/whiteout semantics, advertise non-image config media types, and pull in artifact-only concerns like cosign that don't belong on the image path.

---

Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

[joaopapereira]

I agree that this is a nice to have implementation in vendir. Also on board with a creation of a new separate resource type given the specificities of the artifacts themselves.
I question that I have is what are we going to extract, because if I remember correctly the artifact can point to different images, so should we allow only 1 image extraction per source?
Also what options should we allow the users to select?
When you talk about cosign signatures, does that mean that vendir will have to validate these signatures?

[onedr0p]

should we allow only 1 image extraction per source?

Yeah, one manifest per source. If the tag points to an index, make the user pick with a selector (platform, artifactType, annotations), same way image: handles multi-arch.

Inside that manifest, pull all layers matching mediaTypes, or all of them if no filter. Usually just one layer, but flux and a few others split things up.

Not every layer is a tarball though. WASM and OPA bundles ship as raw blobs, so the extractor needs two modes: untar for tar/tar+gzip, write the blob to disk otherwise (use the org.opencontainers.image.title annotation as the filename, fall back to digest).

what options should we allow the users to select?

image, tag/digest, mediaTypes, artifactType, platform, secretRef (reuses image: auth), dangerousSkipTLSVerify, and cosign.

does that mean that vendir will have to validate these signatures?

Yep. In-process with sigstore-go, not shelling out. Opt-in through the cosign block: publicKey for keyed, certificateIdentity + certificateOidcIssuer for keyless, plus rekorURL/tufMirror for non-default trust roots.

If pulling in sigstore-go feels too heavy for the first iteration it's fine to ignore for now and maybe add it later?

[joaopapereira]

should we allow only 1 image extraction per source?

Yeah, one manifest per source. If the tag points to an index, make the user pick with a selector (platform, artifactType, annotations), same way image: handles multi-arch.

Inside that manifest, pull all layers matching mediaTypes, or all of them if no filter. Usually just one layer, but flux and a few others split things up.

Not every layer is a tarball though. WASM and OPA bundles ship as raw blobs, so the extractor needs two modes: untar for tar/tar+gzip, write the blob to disk otherwise (use the org.opencontainers.image.title annotation as the filename, fall back to digest).

I forgot how all this works, it as been a couple of years since I looked into it.
In terms of convension we can assume if it as tar or tar+gzip it should be untared to disk if not just get the blobs. Do you know if this is something that oras already does by itself or if we will get the bytes and we will have to handle them?

image, tag/digest, mediaTypes, artifactType, platform, secretRef (reuses image: auth), dangerousSkipTLSVerify, and cosign.

This looks like a good list of options, There might be some cases that we will have to handle like mismatch of the artifact type or even lack of particular media type that you are asking for.

If pulling in sigstore-go feels too heavy for the first iteration it's fine to ignore for now and maybe add it later?

This was my concern is that sigstore-go is quite a big thing specially if we are only going to use it in the OCI artifacts. Eventually we can add it and also use it for the Images or other sources.

[joaopapereira]

I just validated and oras-go does not extract layers, it hands back raw blob bytes; v2's file Store only unpacks when the io.deis.oras.content.unpack annotation is present (which third-party artifacts won't have). Vendir owns extraction, and can reuse the existing pkg/vendir/fetch/archive.go extractor, ideally on top of go-containerregistry (already vendored) with no new oras-go dependency.

In terms of the Mismatch/absence → fail loudly (artifactType mismatch errors; no matching mediaType errors).

In terms of the implementation I am a bit torn because oras-go looks like it is in the middle of a rewrite and the latest release of v2 was a year ago, so I am not sure if we need the library itself, maybe when they are more stable we get it in the fold if we think that it makes sense. The majority of the interaction already exist internally because we do have ggcr and imgpkg does some heavy lifting in the OCI registry side so maybe we can just pick the specific bits of the artifacts from the oras-go code and add it to our source.