Skip to content

feat(skill-generator): per-provider SDK version-tracking via sdks field#59

Merged
leggetter merged 1 commit into
mainfrom
feat/per-provider-sdk-versions
May 11, 2026
Merged

feat(skill-generator): per-provider SDK version-tracking via sdks field#59
leggetter merged 1 commit into
mainfrom
feat/per-provider-sdk-versions

Conversation

@leggetter
Copy link
Copy Markdown
Collaborator

Summary

Lets each provider declare its own SDK packages in providers.yaml so the version-staleness check during generation and review actually covers provider SDKs — not just the generic framework deps (Express, FastAPI, vitest, etc.) the generator was previously hardcoded to query.

Why

Caught while reviewing PR #58. The "Dependencies" check in review-skill.md (line 82) tells the AI to grade pins against {{VERSIONS_TABLE}}. That table is built from a hardcoded list in scripts/skill-generator/lib/versions.ts:

const NPM_PACKAGES = ['next', 'express', 'vitest', 'jest', 'typescript'];
const PIP_PACKAGES = ['fastapi', 'pytest', 'httpx'];

So when reviewing paddle-webhooks, the AI saw current versions for Express/Next.js/TypeScript and correctly flagged typescript ^5.9.3 (minor — 1 major behind). But @paddle/paddle-node-sdk was invisible — the table never mentioned it, so the AI had no signal that ^1.4.0 was 2 majors behind ^3.8.0. That gap is what produced the stale Paddle SDK reference in PR #58 (and was also why the original paddle-webhooks generation pinned ^1.4.0 to begin with).

Schema addition

providers.yaml entries gain an optional sdks field:

- name: paddle
  ...
  sdks:
    npm:
      - "@paddle/paddle-node-sdk"
    pip:
      - paddle-python-sdk

Field choice (sdks vs packages): sdks is more semantically specific — these are the provider's own SDK packages we want version-tracked. The generic framework deps stay covered by the hardcoded list in versions.ts.

Documented at the top of providers.yaml alongside the existing name/displayName/docs/notes/testScenario schema.

Plumbing

  • types.tsProviderConfig.sdks?: { npm?: string[]; pip?: string[] }
  • config.ts — parser reads sdks from both the array and object yaml formats
  • versions.ts:
    • Exports the generic NPM_PACKAGES / PIP_PACKAGES lists
    • getLatestVersions(extras?: { npm; pip }) merges and de-dupes generic + per-provider lists before querying in parallel
    • formatVersionsTableForProvider(versions, sdks) filters the global cache down to generic deps + this provider's SDKs (avoids dumping every queried SDK across all providers into every prompt)
    • collectProviderSdks(providers) unions all SDKs across a config set so index.ts can pre-fetch the full superset once
  • cli.tsbuildPromptReplacements uses the per-provider table
  • index.ts — both generate and review call sites pass collectProviderSdks(providerConfigs) to the version query

Seeded paddle only

The other 30+ providers' sdks entries can be added in a follow-up sweep — separating the mechanism from the data keeps this PR small and focused.

Test result

Verified end-to-end. With paddle.sdks populated, the generator's startup log now reports:

Querying package managers for latest stable versions...
  npm: next@16.2.6, express@5.2.1, vitest@4.1.6, jest@30.4.2, typescript@6.0.3, @paddle/paddle-node-sdk@3.8.0
  pip: fastapi@0.136.1, pytest@9.0.3, httpx@0.28.1, paddle-python-sdk@1.14.1

Then running ./scripts/generate-skills.sh review paddle --config providers.yaml --model claude-opus-4-7 from this branch flagged exactly the issues we'd hope for:

  • major@paddle/paddle-node-sdk ^1.4.0 in examples/express/package.json is 2 majors behind ^3.8.0
  • major — same in examples/nextjs/package.json
  • minorpaddle-python-sdk>=1.0.0 in examples/fastapi/requirements.txt is too loose; tighten to >=1.14.1 (the loose pin permits versions that predate the Verifier/Secret API the docs reference)

The review iteration applied those bumps (worked output sitting on the local improve/paddle-webhooks branch; happy to push it as either a fresh PR or on top of PR #58 if that's helpful — see comment below).

Test plan

  • cd scripts/skill-generator && npx tsc --noEmit — typechecks clean
  • Smoke-test the helper end-to-end (the test script in the PR's commit message reproduces it)
  • Run ./scripts/generate-skills.sh review paddle --config providers.yaml and confirm the version table in the prompt now includes both @paddle/paddle-node-sdk and paddle-python-sdk
  • Run review on a provider without sdks declared and confirm behavior is unchanged (no SDK rows in the table, generic deps still queried)

https://claude.ai/code/session_01NNTgQRJss1V7gyzzJ9rjnB


Generated by Claude Code

…ield

The version-staleness check in the generator/reviewer was only seeing
generic framework deps (Express, FastAPI, vitest, etc.) — provider-
specific SDKs were invisible, so heavy-pin drift like Paddle's
`@paddle/paddle-node-sdk ^1.4.0` (2 majors behind 3.8.0) didn't get
flagged during review. Fixes that by letting each provider declare its
SDK packages in `providers.yaml` and threading them through to the
version-query batch.

Changes:

- `providers.yaml` schema gains an optional `sdks: { npm: [...], pip: [...] }`
  field per provider, documented in the file header.
- `scripts/skill-generator/lib/types.ts` — `ProviderConfig.sdks` added.
- `scripts/skill-generator/lib/config.ts` — parser reads `sdks` from both
  the array and object yaml formats.
- `scripts/skill-generator/lib/versions.ts`:
  - Exports `NPM_PACKAGES` / `PIP_PACKAGES` (the generic lists).
  - `getLatestVersions` takes an optional `extras: { npm, pip }` arg and
    merges + de-dupes the lists before querying in parallel.
  - New `formatVersionsTableForProvider(versions, sdks)` filters the
    global cache down to generic deps + this provider's SDKs, so the
    prompt doesn't dump every queried SDK across all providers.
  - New `collectProviderSdks(providers)` unions all SDKs across a
    config set so `index.ts` can pre-fetch the full superset once.
- `scripts/skill-generator/lib/cli.ts` — `buildPromptReplacements` now
  uses the per-provider table.
- `scripts/skill-generator/index.ts` — both generate and review call
  sites pass `collectProviderSdks(providerConfigs)` to the version query.

Seeded `paddle`'s entry as the test case:

    sdks:
      npm:
        - "@paddle/paddle-node-sdk"
      pip:
        - paddle-python-sdk

Other providers' SDK declarations can be added in a follow-up sweep.

Smoke-tested end-to-end: with paddle's `sdks` populated, the version
table emitted for the paddle prompt contains
`@paddle/paddle-node-sdk@3.8.0` and `paddle-python-sdk@1.14.1`
alongside the generic deps. Re-running review on paddle (with the
existing `^1.4.0` pins on main) should now flag those as 2 majors
behind under the existing severity rubric and propose the bump.

https://claude.ai/code/session_01NNTgQRJss1V7gyzzJ9rjnB
leggetter added a commit that referenced this pull request May 11, 2026
…y-protection doc

- Bump @paddle/paddle-node-sdk to ^3.8.0 in express and nextjs examples
- Bump nextjs typescript to ^6.0.3, next to ^16.2.6
- Tighten paddle-python-sdk to >=1.14.1 to match documented Verifier/Secret API
- Remove replay-protection gotcha from verification.md so docs match handlers
  (strict 5s window would reject Paddle's legitimate retries)

Produced by re-running `./scripts/generate-skills.sh review paddle` after
the per-provider SDK version-tracking feature (PR #59) was applied. The
review iteration's prompt now sees @paddle/paddle-node-sdk@3.8.0 and
paddle-python-sdk@1.14.1 in {{VERSIONS_TABLE}}, applies the existing
severity rubric (2 majors behind → major), and proposes the bumps.

https://claude.ai/code/session_01NNTgQRJss1V7gyzzJ9rjnB
@leggetter leggetter marked this pull request as ready for review May 11, 2026 21:39
@leggetter leggetter merged commit 9e5a653 into main May 11, 2026
5 checks passed
@leggetter leggetter deleted the feat/per-provider-sdk-versions branch May 11, 2026 21:39
leggetter added a commit that referenced this pull request May 11, 2026
…utput) (#58)

* fix(paddle-webhooks): trim SKILL.md to verification core, correct SDK versions

- Trim SKILL.md Express/FastAPI snippets to verification-core only,
  matching the stripe-webhooks convention (drops route wiring, event
  dispatch, response handling)
- Fix SDK version claims in references/verification.md
  (@paddle/paddle-node-sdk v1.4.0+; paddle-python-sdk v1.14.0+)
- Remove dead Verifier branch in examples/fastapi/main.py
  (SDK targets Flask/Django request objects, FastAPI verifies manually)
- Bump fastapi/requirements.txt to current stable floors
- Remove orphan "# Or via NPM" comment from Local Development section

https://claude.ai/code/session_01NNTgQRJss1V7gyzzJ9rjnB

* fix(paddle-webhooks): bump SDKs and tooling, drop unimplemented replay-protection doc

- Bump @paddle/paddle-node-sdk to ^3.8.0 in express and nextjs examples
- Bump nextjs typescript to ^6.0.3, next to ^16.2.6
- Tighten paddle-python-sdk to >=1.14.1 to match documented Verifier/Secret API
- Remove replay-protection gotcha from verification.md so docs match handlers
  (strict 5s window would reject Paddle's legitimate retries)

Produced by re-running `./scripts/generate-skills.sh review paddle` after
the per-provider SDK version-tracking feature (PR #59) was applied. The
review iteration's prompt now sees @paddle/paddle-node-sdk@3.8.0 and
paddle-python-sdk@1.14.1 in {{VERSIONS_TABLE}}, applies the existing
severity rubric (2 majors behind → major), and proposes the bumps.

https://claude.ai/code/session_01NNTgQRJss1V7gyzzJ9rjnB

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants