feat: CRD value accessors (native modules + raw objects, src + chart)#99
Draft
sini wants to merge 6 commits into
Draft
feat: CRD value accessors (native modules + raw objects, src + chart)#99sini wants to merge 6 commits into
sini wants to merge 6 commits into
Conversation
fromCRD now accepts an optional `kindFilter` (a list of CRD `kind` names). When set, only CustomResourceDefinitions whose kind is in the list are generated; an empty/unset filter (the default) keeps every CRD, so the change is fully backwards compatible. This helps when `crds` points at a multi-document stream — e.g. raw `helm template` output — that contains more kinds than you want to expose. To support that, crd2jsonschema.py also skips null documents (empty or comment-only YAML docs, which yaml.safe_load_all yields as None) instead of raising. Adds unit tests covering kind filtering, the empty-filter passthrough, and null-document skipping.
…ound-trip
`fromCRD` generates Nix *source*, writes a `.nix` file, and you `import` it
back — a serialize/deserialize round-trip plus an import-from-derivation. For
flake consumers that just want the resource options, that intermediate file
is pure overhead.
`fromCRDModule` returns the resource definitions as a module *value*
(`{ lib, options, config, ... }: { ... }`) instead. It drops straight into
`nixidy.applicationImports` (which already accepts `functionTo attrs`), so
there is no generated file and no import IFD — only the unavoidable Python
crd2jsonschema parse, now shared between both backends via the extracted
`crdSchema` helper.
- runtime.nix: the per-module boilerplate that generator.nix inlines as
source (submoduleOf, coerce-attrs-to-list-by-key, the str/coercedTo type
overrides, defaults handling, ...), extracted as real functions.
- module.nix: a value-emitting port of generator.nix's schema walk
(genDefinitions / mapType / genResourceOptions).
- equiv-test.nix + `nix run .#crdModuleTest`: asserts fromCRDModule renders
identically to fromCRD for a CRD that exercises submodules, list-by-key
coercion and the global ObjectMeta metadata ref.
The text generator (`fromCRD`) is unchanged — verified to produce
byte-identical output — and remains for the CLI "commit generated types"
workflow, which needs source files.
The fromCRDModule prototype duplicated generator.nix's schema walk
(genDefinitions / mapType / genResourceOptions) inside module.nix — two
hand-kept twins of the type/coercion branch logic, the part most likely to
drift (flagged in review).
Extract the walk into walk.nix, parameterized by a `backend` of emit
combinators (types, mkOption, submoduleOf, coerce-attrs-to-list-by-key, ...).
Two backends implement it:
- backend-text.nix -> Nix source fragments; generator.nix assembles them
into the committable standalone .nix file
- backend-value.nix -> live values via runtime.nix; module.nix assembles
them into a module value
generator.nix and module.nix are now thin assemblers over the one walk; the
branch logic exists in a single place (net -197 lines).
Verified: file generator output byte-identical (argocd store hash unchanged);
fromCRDModule still renders identically to fromCRD (nix run .#crdModuleTest);
moduleTests 42/42; crd2jsonschema 18/18; staticCheck clean.
Remaining (irreducible) duplication: the runtime helper block exists as live
functions in runtime.nix and as inlined source in generator.nix's template.
Committed standalone files can't reference nixidy internals, and live
types/functions can't be serialized back to source — so the two forms must
coexist. The walk (the real divergence risk) is no longer duplicated.
crdObjects { src, crds, kindFilter ? [ ] } returns the raw
CustomResourceDefinition objects from a set of CRD YAML files as values —
the objects counterpart to fromCRD's types. Deployment-agnostic: it hands
back the manifests and lets the caller decide what to do with them (e.g.
apply them to a cluster), rather than baking in an install mechanism.
Reuses fromChartCRD's isWanted filter shape; kindFilter mirrors fromCRD's
(empty = every CRD).
Add the chart counterparts of the src-based accessors:
- fromChartCRDModule → resource type module from a chart's CRDs
- crdObjectsFromChart → raw CustomResourceDefinition manifests from a chart
Both share an internal mkChartCRDsYaml (raw `helm template --include-crds`),
so calling both with identical args templates the chart once. The output is
raw helm YAML parsed directly downstream — no klib.fromHelm/yq-go
re-serialization, avoiding its number-coercion quirks.
kubeVersion is now a first-class, overridable argument across the chart
family (fromChartCRD too), defaulting to nixidy's nixpkgs version. Consumers
templating CRDs for a cluster on a different Kubernetes version can target it
explicitly — previously the version was hardcoded to nixidy's pkgs.
Completes the CRD API matrix: {fromCRD, fromCRDModule, crdObjects} × {src,
chart}. fromChartCRD is unchanged for existing callers (same default).
Expand the accessor test (renamed crdModuleTest -> crdAccessorTest) from fromCRD<->fromCRDModule equivalence to the full value-accessor surface: - fromChartCRDModule renders identically to fromCRDModule for the same CRD (a local helm chart shipping the CRD in crds/, copied verbatim). - crdObjects returns the raw CRD manifests; kindFilter narrows (hit + miss). - crdObjectsFromChart returns the same objects as crdObjects. - the chart accessors honor kubeVersion: a templated CRD embedding .Capabilities.KubeVersion renders v1.31 vs v1.40 observably differently. Seven checks; the derivation builds iff all pass.
sini
changed the title
Contributor
Author
|
Using this here to automatically inject CRDs into evaluation for type checking/defaults, without maintaining static CRD nix files. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Opening as a draft for discussion — this is a sizable change to the
generators and I'd rather get your read on the direction before polishing.
Summary
Adds CRD generators that return Nix values instead of building a generated
.nixfile, plus their chart and raw-object counterparts — completing the APImatrix:
fromCRDfromChartCRDfromCRDModulefromChartCRDModulecrdObjectscrdObjectsFromChartMotivation
fromCRDrenders the resource options to Nix source,nixfmts it, writes a.nixfile, and youimportit back — a serialize→deserialize round-trip plusan import-from-derivation. For flake/programmatic consumers that just want the
options, that file is pure overhead.
nixidy.applicationImportsalready acceptsfunctionTo attrs, sofromCRDModulereturns the module value directly anddrops straight in — no file, no import IFD.
crdObjects/crdObjectsFromChartexpose the rawCustomResourceDefinitionmanifests as values (e.g. to apply them to a cluster), deliberately
deployment-agnostic — they hand back objects and let the caller decide.
What's in here
fromCRDModule— native module value; no generated file.walk.nix, driving a text backend (source, for the committed-file workflow) and a value backend (live module).generator.nix/module.nixbecome thin assemblers over the one walk, so the branch logic lives in a single place.crdObjects— raw CRD manifests from asrc.fromChartCRDModule/crdObjectsFromChart, sharing one memoizedhelm template;kubeVersionis now overridable across the chart family (incl.fromChartCRD), defaulting to nixidy's nixpkgs version — previously hardcoded.nix run .#crdAccessorTest).Backwards compatibility / verification
fromCRD/fromChartCRDfile output is unchanged — verifiednix build .#generators.argocdyields the same store path before/after the backend refactor.fromChartCRDonly gains an optionalkubeVersionwith its prior default.moduleTests42/42,crd2jsonschemaTest18/18,crdAccessorTest(7 checks incl. chart↔src equivalence andkubeVersionhonored),staticCheck.TODO before this is merge-ready
llms.txtentries forfromCRDModule,crdObjects,fromChartCRDModule,crdObjectsFromChart, and thekubeVersionargument.fromCRDModuleviaapplicationImports(andcrdObjectsto deploy the CRDs), showing the no-generated-files workflow.mainonce feat: add optional kindFilter to fromCRD #98 lands.Notes for review
helpers exist both as source (inlined by the text backend, so committed
standalone files stay self-contained) and as values (
runtime.nix, for thenative modules). Live types/functions can't be serialized back to source, and
committed files can't reference nixidy internals — so the two forms must
coexist. The schema walk (the real divergence risk) is single-sourced.