Wire elastic-package to package-spec validation modes by teresaromero · Pull Request #3617 · elastic/elastic-package

teresaromero · 2026-05-28T10:36:12Z

Problem

elastic-package calls package-spec validation at several points in the lifecycle of a package — during lint (source tree), build (built artifact, directory and zip), and install --zip (pre-built zip). All of these call sites were using the same deprecated ValidateFrom* / ValidateAndFilter* functions, which run in ModeLegacy. This means the validation rules applied were identical regardless of whether the package was a source checkout or a built artifact.

Why

package-spec now exposes explicit validation modes (ModeSource and ModeBuild) that reflect what a package actually is at each phase:

ModeSource allows source-only artifacts: _dev/ directories, .link files, and external: ecs field references. These are normal in a checked-out package and are resolved or stripped during the build step.
ModeBuild rejects those same artifacts. A built package that still contains _dev/ or unresolved external: ecs references is incorrect and should fail validation.

Running lint with ModeLegacy (instead of ModeSource) could silently allow or reject things inconsistently with what the source phase actually permits. Running build and install with ModeLegacy meant that built artifacts carrying source-only remnants would not be caught.

Changes

The internal/validation package is rewritten to expose three named functions, one per validation context: ValidateSourceFromPath, ValidateBuiltFromPath, and ValidateBuiltFromZip. The mode selection is encapsulated inside the validation package — call sites never reference validator.Mode directly. Dead code (ValidateFromPath, ValidateFromZip) with zero callers is removed.

All four call sites are updated to use the context-appropriate function: lint uses ValidateSourceFromPath; build uses ValidateBuiltFromPath and ValidateBuiltFromZip; install --zip uses ValidateBuiltFromZip.

Tests are added to internal/validation covering the happy path for each new function. TestValidateBuiltFromPath and TestValidateBuiltFromZip each include a rejection case confirming that a source package containing _dev/ is rejected — guarding against silent reversion to ModeLegacy behaviour on both the directory and zip code paths. A makeZipFromDir test helper generates zip fixtures on-the-fly, so no binary test fixture is committed.

ValidateBuiltFromZip also gains logger.Debugf calls in the two previously silent error paths (zip open failure and subdirectory extraction failure), making them visible under --verbose and consistent with the logging in validateFromPath.

Dependency

This PR depends on elastic/package-spec#1178 and package-spec being released

Testing

Tested by running a local build against the integrations packages. After the change, all packages validate as expected with no new failures and no regressions.

Resolves [Change Proposal] Add different validation modes for source and built packages package-spec#549

…h, ValidateBuiltFromZip Add three new exported functions that use the package-spec ModeSource and ModeBuild constructors instead of the deprecated ValidateFrom* free functions. Existing ValidateAndFilter* functions are kept untouched until callers are migrated. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Replace ValidateAndFilterFromPath/ValidateAndFilterFromZip with ValidateBuiltFromPath and ValidateBuiltFromZip in builder and installer packages to use the new named validation mode functions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…Filter* functions All call sites have been migrated to the named ValidateSource* and ValidateBuilt* functions, so the legacy wrappers are no longer referenced. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Four table-driven test functions (ValidateSourceFromPath, ValidateBuiltFromPath, ValidateBuiltFromZip) with local testdata fixtures covering both happy-path and cross-mode rejection cases: - source_package: has _dev/, passes ModeSource, fails ModeBuild (ValidateNoDevFolder) - built_package: has _embedded_ecs dynamic templates, passes ModeBuild, fails ModeSource (ValidateNoEmbeddedEcsInDynamicTemplates) - composable_source_package: uses package: stream references, passes ModeSource, fails ModeBuild (ValidateStreamInputMaterialized) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

This replace is only needed during local development when working against an unpublished package-spec checkout. It must not be committed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

The PR branch (teresaromero/package-spec@9c3e7024) cannot be fetched by the module proxy because the /v3 module path prevents Go from resolving fork commits until the PR is merged and the proxy indexes them. Replace with the local checkout, which is already at that exact commit. Remove this directive and bump the require version once PR elastic#1175 merges. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- Extract validateFromPath helper to deduplicate ValidateSourceFromPath and ValidateBuiltFromPath (identical bodies, different mode constant). - Return allErrors unfiltered instead of dropping them when filterErrors fails; log the infrastructure error at debug level so it is not silently lost. - In ValidateBuiltFromZip, return allErrors (unfiltered) if the second zip.OpenReader or fsFromPackageZip call fails, preserving the validation result in all error paths. - Fix fsFromPackageZip to count only directory entries at the zip root; the previous check counted all entries, so a root-level non-directory file (e.g. __MACOSX metadata) would falsely trigger 'single directory expected'. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

teresaromero · 2026-05-28T13:58:46Z

 	return subDir, nil
 }

+// TODO: follow-up issue — move this logic into specerrors as an exported function so consumers don't reimplement it.


filed issue #3618

…lidation-modes

Updated the validation functions to use more descriptive mode constants. Changed `ModeSource` to `SourceMode` and `ModeBuild` to `BuildMode`. Improved error handling in the validator creation process to provide clearer error messages. Adjusted validation calls to use the appropriate methods for path and zip validation.

github-actions · 2026-06-16T14:47:11Z

TL;DR

All three failed Buildkite jobs are blocked before meaningful static checks/tests run because the PR adds a local Go module replacement for github.com/elastic/package-spec/v3 to ../package-spec, but that sibling checkout does not exist on CI.

Remediation

Remove the local replace github.com/elastic/package-spec/v3 => ../package-spec before relying on CI, and pin github.com/elastic/package-spec/v3 to a version/commit that CI can fetch.
Re-run make check-static and make test-go-ci after updating go.mod/go.sum.

Investigation details

Root Cause

This is a dependency/configuration failure introduced by the PR's go.mod change. The PR diff adds:

// go.mod:59-61
// Local replace pointing to elastic/package-spec PR #1175 (teresaromero/package-spec@9c3e7024).
// Go cannot fetch fork commits for /v3 modules until the PR is merged and the proxy indexes it.
replace github.com/elastic/package-spec/v3 => ../package-spec

Buildkite checks out only elastic/elastic-package, so ../package-spec is absent. Go then fails module resolution anywhere the package-spec module is imported, including internal/validation/validation.go and internal/packages/archetype/spec.go.

Evidence

Build: https://buildkite.com/elastic/elastic-package/builds/8202
Failed jobs: :go: Run check-static, :linux: Run unit tests, :macos: Run unit tests
Key log excerpt from make check-static:

internal/validation/validation.go:15:2: github.com/elastic/package-spec/v3@v3.6.3: replacement directory ../package-spec does not exist
internal/validation/validation.go:16:2: github.com/elastic/package-spec/v3@v3.6.3: replacement directory ../package-spec does not exist
internal/packages/archetype/spec.go:12:2: github.com/elastic/package-spec/v3@v3.6.3: replacement directory ../package-spec does not exist
make: *** [Makefile:23: build] Error 1

The Linux and macOS unit-test jobs fail with the same replacement directory ../package-spec does not exist error and then report many downstream package failures because compilation cannot proceed.

Verification

Not run locally against the PR head; the local checkout is at the base commit, and the CI logs plus PR diff directly identify the unresolved local replacement.

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

elasticmachine · 2026-06-16T15:11:07Z

💔 Build Failed

Buildkite Build
Commit: acda4eb

Failed CI Steps

History

💔 Build #8202 failed 4f5faa4
💔 Build #8085 failed 8703b2b

…lidation-modes

teresaromero · 2026-06-22T08:24:52Z

 	result, err := filterErrors(allErrors, fsys)
 	if err != nil {
-		return err, nil
+		logger.Debugf("failed to filter validation errors: %v", err)


at this point allErrors is not nil; so in case any error happens during the filering, they were ignored... i've added a debug log with the error and return allErrors unfiltered; same un the zip case

mrodm

LGTM!
Tested locally to force failures in build and source modes and elastic-package lint and elastic-package build commands detected those.

Thanks!

teresaromero and others added 10 commits May 28, 2026 11:21

Update .gitignore to include .scratch directory

a73ef71

cmd/lint: migrate validateSourceCommandAction to ValidateSourceFromPath

4942eac

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

internal/validation: document error filtering in godoc for new functions

b1e1a84

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

go.mod: remove local package-spec replace directive

3849d7b

This replace is only needed during local development when working against an unpublished package-spec checkout. It must not be committed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

teresaromero requested a review from a team as a code owner May 28, 2026 10:36

teresaromero marked this pull request as draft May 28, 2026 10:36

teresaromero mentioned this pull request May 28, 2026

Add source and build validation modes elastic/package-spec#1175

Closed

2 tasks

teresaromero commented May 28, 2026

View reviewed changes

teresaromero added 4 commits June 2, 2026 14:15

Merge branch 'main' of github.com:elastic/elastic-package into 549-va…

d91ca7e

…lidation-modes

Merge branch 'main' of github.com:elastic/elastic-package into 549-va…

232d6c6

…lidation-modes

Remove .scratch entry from .gitignore

4f5faa4

Enhance validation tests for built packages

acda4eb

teresaromero marked this pull request as ready for review June 16, 2026 15:07

teresaromero added 3 commits June 22, 2026 10:13

Merge branch 'main' of github.com:elastic/elastic-package into 549-va…

b98f50d

…lidation-modes

chore: remove local replace directive for package-spec in go.mod

90099fe

fix: restore logger import in validation.go

f243dad

teresaromero commented Jun 22, 2026

View reviewed changes

Merge branch 'main' into 549-validation-modes

3a1e196

mrodm approved these changes Jun 23, 2026

View reviewed changes

teresaromero merged commit be05aeb into elastic:main Jun 23, 2026
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Wire elastic-package to package-spec validation modes#3617

Wire elastic-package to package-spec validation modes#3617
teresaromero merged 19 commits into
elastic:mainfrom
teresaromero:549-validation-modes

teresaromero commented May 28, 2026 •

edited

Loading

Uh oh!

teresaromero May 28, 2026

Uh oh!

github-actions Bot commented Jun 16, 2026

Root Cause

Evidence

Verification

Uh oh!

elasticmachine commented Jun 16, 2026 •

edited

Loading

Uh oh!

teresaromero Jun 22, 2026

Uh oh!

mrodm left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

teresaromero commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Why

Changes

Dependency

Testing

Uh oh!

teresaromero May 28, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jun 16, 2026

TL;DR

Remediation

Root Cause

Evidence

Verification

Uh oh!

elasticmachine commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💔 Build Failed

Failed CI Steps

History

Uh oh!

teresaromero Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

mrodm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

teresaromero commented May 28, 2026 •

edited

Loading

elasticmachine commented Jun 16, 2026 •

edited

Loading