Jsii migration#47
Draft
ms280690 wants to merge 7 commits into
Draft
Conversation
* feat: scaffold jsii TypeScript project for polyglot SDK
Add prescient-sdk-ts/ subdirectory with jsii-compatible TypeScript
project structure. Generates native packages for Python, C#/.NET,
Go, Java, and TypeScript/npm from a single TypeScript source.
- package.json with jsii config for all 5 language targets
- pnpm as package manager
- jest + ts-jest for testing
- Minimal src/index.ts and src/types.ts that compile clean
- JSII_MIGRATION_PLAN.md documents full migration plan
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(phase-1): address critical review findings before merge
- typescript devDep bumped from ~5.4 to ~5.9 to match jsii@5.x requirement
- @types/node bumped from ^18 to ~22 for Node 22 runtime
- PyPI distName renamed prescient-sdk → prescient-sdk-sparkgeo to avoid
collision with live PyPI package v0.4.1
- repository.type field added ("git")
- .jsii and tsconfig.json untracked (auto-generated by jsii on every build)
- .gitignore updated: add .jsii, tsconfig.json, .env patterns; scope
*.js glob to dist/**/ and targets/**/ to avoid shadowing jest.config.js
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(phase-2): define public API types and structs Adds all jsii-safe types derived from the Python SDK: - AuthProvider enum (MICROSOFT | GOOGLE) - PrescientClientOptions struct — googleClientSecret intentionally absent; consumers must set PRESCIENT_GOOGLE_CLIENT_SECRET env var to prevent the secret from appearing in jsii IPC logs - AuthCredentials, BucketCredentials, RequestHeaders, UploadOptions structs - All Date fields replaced with ISO 8601 string (expiresAt) per jsii constraint - 9 Jest tests covering required/optional field shapes and enum values Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(phase-2): address all review findings before merge types.ts: - BucketCredentials: add @remarks security warnings on secretAccessKey and sessionToken — discard after constructing native SDK client, do not log; document jsii IPC serialization risk for JSII_DEBUG=1 users - AuthCredentials.refreshToken: document long-lived nature and Google once-only issuance; warn against logging - RequestHeaders.authorization: warn bearer token must not be logged - endpointUrl / authUrl: document HTTPS requirement (enforcement deferred to Phase 3 settings validation) - UploadOptions.inputDir: document path traversal responsibility; note implementation will reject .. traversal (enforcement deferred to Phase 5) types.test.ts: - Replace 'AKIA...', 'secret', 'token' fixtures with unambiguous PLACEHOLDER values that won't trip secret scanner rules - Add gitleaks:allow comments on credential-shaped fixture lines - Replace 'Bearer token' with 'Bearer PLACEHOLDER_TOKEN' package.json: - Remove dotenv from bundledDependencies and dependencies — unused in Phase 2 (types only); re-add in Phase 3 when settings.ts calls dotenv.config() jest.config.js: - Add testPathIgnorePatterns to exclude dist/ and targets/ — jsii build emits dist/__tests__/types.test.d.ts which Jest was incorrectly picking up ci.yaml / publish.yaml / deploy-book.yaml: - Add top-level permissions: {} deny-all default on all three workflows - ci.yaml: remove unused id-token: write and pages: write; scope to contents: read - publish.yaml: move TWINE_PASSWORD from workflow-level env to publish step only; remove unused id-token: write and pages: write; add environment: pypi-production gate; restrict tag glob to semver v-prefix (v[0-9]+.[0-9]+.[0-9]+) - deploy-book.yaml: pages: write + id-token: write retained (required by actions/deploy-pages); restrict tag glob to semver v-prefix - SHA-pin all action uses: to full commit hashes dependabot.yml: - Add Dependabot config for github-actions (weekly) and npm /prescient-sdk-ts (weekly) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(phase-3): add Settings class with env var loading and validation - Settings class reads PRESCIENT_* env vars, falling back to PrescientClientOptions when opts are passed directly (opts take precedence over env) - HTTPS enforced on endpointUrl and authUrl at construction time — throws a descriptive error for http:// or any non-HTTPS scheme (Phase 2 blocker resolved) - Required-field validation: endpointUrl, clientId, authUrl always required; tenantId required for MICROSOFT provider; PRESCIENT_GOOGLE_CLIENT_SECRET env var required for GOOGLE provider - _googleClientSecret stored as @internal _-prefixed field — stripped from jsii assembly so it never appears in Python/Go/C#/Java bindings or crosses IPC - googleRedirectPort defaults to 8765; validates env var as integer 1–65535 - PRESCIENT_AUTH_PROVIDER parsed case-insensitively from env - 17 Jest tests covering happy paths, env var precedence, and all validation errors Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Address PR #45 review findings: SSRF blocklist, port validation, toJSON, CI - assertNotSsrf(): block IMDS (169.254.169.254), localhost, loopback, RFC 1918 (10/127/172.16-31/192.168) on endpointUrl and authUrl - resolvePort(): validate fromOpts range 1-65535; use parseInt(n,10) + String(n) cross-check to reject scientific notation / hex / trailing garbage from env var - assertHttps(): sanitize error message via new URL(url).origin to strip embedded credentials before logging - Settings.toJSON(): explicit allowlist returning PrescientClientOptions, excludes _googleClientSecret from JSON.stringify output - awsRole ARN prefix + length (≤ 2048) validation - New tests: port boundary (0, 65536, 1.5, "1e4"), SSRF (IMDS/localhost/ RFC1918), toJSON secret exclusion, awsRole ARN — 37 tests total - ci.yaml: add typescript-sdk job (pnpm SHA-pinned, --frozen-lockfile, build + test); timeout-minutes on both jobs - .gitleaks.toml: allowlist for test fixture placeholder values - gitleaks:allow inline comments on secret-shaped test fixture lines Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Phase 4: PrescientClient — auth, credential caching, STS, fileproxy
PrescientClient with feature parity to the Python SDK:
- Microsoft MSAL: acquireTokenInteractive (browser) on first call,
acquireTokenSilent (cached AccountInfo) on subsequent calls
- Google OAuth2: InstalledAppFlow equivalent — local HTTP server on
googleRedirectPort catches the redirect code; subsequent calls use
stored refresh_token via oauth2Client.refreshAccessToken()
- 5-minute timeout on OAuth2 browser flow; server closed on success/error
- openSystemBrowser(): cross-platform (macOS/Linux/Windows) via spawn,
used by both MSAL interactive and Google initial flow
- bucketCredentials(): STS AssumeRoleWithWebIdentity when awsRole set,
otherwise _fetchFileproxyCredentials() (fetch + Bearer token)
- uploadBucketCredentials(): STS with uploadRole; throws if not configured
- credentialsExpired: sync bool property (pre-auth = true)
- refreshCredentials(force?): clears cache on force=true, re-fetches all
- stacCatalogUrl: computed as joinUrl(endpointUrl, 'stac') in constructor
- 1-hour auth TTL (matching Python SDK); bucket creds use STS Expiration
- Method names: bucketCredentials / uploadBucketCredentials (not getXxx —
jsii JSII5000: getXxx conflicts with Java property getters)
- engines: { node: ">=18" } added — required for global fetch API
- 48 tests (11 new client tests); jsii build clean
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* upgrade
Signed-off-by: ms280690 <mehul@sparkgeo.com>
* Address PR #46 review findings: security, concurrency, CI hardening
Security fixes:
- captureOAuthCode: bind HTTP server to 127.0.0.1 (was 0.0.0.0)
- captureOAuthCode: add PKCE (RFC 8252 §8.1) via generateCodeVerifierAsync()
and CodeChallengeMethod.S256; add random state (RFC 6749 §10.12) with
server-side state validation to prevent CSRF / code injection
- captureOAuthCode: HTML-escape OAuth error parameter before reflecting
into error page response body (XSS)
- captureOAuthCode: reject non-GET and non-root-path requests
- openSystemBrowser: resolve on 'spawn' event rather than unconditionally
to avoid race where error event is silently dropped after resolve()
- _fetchStsCredentials: use parts.at(-1) for RoleSessionName stub (correct
for paths like role/path/to/Name); sanitise to [a-zA-Z0-9+=,.@_-] and
truncate to 44 chars (prefix 'prescient-s3-access-' = 20, total ≤ 64)
- _fetchFileproxyCredentials: validate expiration date before use; NaN
expiry caused isExpired() to always return false → stale creds never evict
- types.ts: add jsii IPC / JSII_DEBUG=1 warning to AuthCredentials.refreshToken
Correctness fixes:
- authenticate(): in-flight promise deduplication via _authInFlight; same
pattern applied to bucketCredentials() / uploadBucketCredentials() —
concurrent callers share one browser flow instead of racing on the port
- authenticate(): remove expiresAt override; trust provider-reported TTL
from MSAL expiresOn / Google expiry_date instead of always capping at 1hr
- _getStsClient(): cache STSClient instead of re-instantiating per call
CI hardening:
- ci.yaml: add concurrency group with cancel-in-progress
- ci.yaml: fail-fast: false on Python matrix (show all broken versions)
- ci.yaml: pnpm/action-setup — remove version: pin, let packageManager
field drive pnpm version (was 11.1.2, packageManager declares 11.5.0)
- ci.yaml: pnpm install --ignore-scripts to suppress postinstall lifecycle
hooks from transitive dependencies during CI
- ci.yaml: add cache-dependency-path comment (relative to repo root)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Signed-off-by: ms280690 <mehul@sparkgeo.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(phase-5): Uploader class — directory upload to S3 with multipart and exclude patterns
- Uploader.upload() walks inputDir recursively, keys files as <dir-name>/<relative-path>
- Glob exclusion mirrors Python pathlib.Path.match: basename-only patterns for no-slash, full relative path for slash patterns
- overwrite=false issues HeadObject before Upload; 404 proceeds, other errors propagate
- Uses @aws-sdk/lib-storage Upload for multipart; stream is destroyed after done()
- 11 unit tests covering upload, exclusion, skip-existing, and error paths
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: use npm pack --ignore-scripts to work around pnpm symlinks
jsii-pacmak defaults to `npm pack <path>` which crashes (npm error: Exit
handler never called) when node_modules uses pnpm's symlinked virtual
store. Passing any non-default --pack-command causes jsii-pacmak to
copy the module with dereference:true first, then run the command in the
resolved copy where npm pack succeeds.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs: add per-target build prerequisites to TypeScript SDK README
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* move
Signed-off-by: ms280690 <mehul@sparkgeo.com>
* fix(ci): pin pnpm version explicitly in action-setup step
pnpm/action-setup reads packageManager from the repo-root package.json,
but ours is in prescient-sdk-ts/package.json — not the root. Explicit
version avoids the lookup failure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci: add language toolchains and package step to typescript-sdk job
Installs Python 3.12, Go 1.22, .NET 8, and Java 21 (Temurin) so that
jsii-pacmak can generate all five language targets. Adds a Package step
after Test and bumps the job timeout to 25 minutes to account for the
dereference copy that pnpm pack requires.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ci): use stable Go version — jsii-pacmak requires go >= 1.25
The Go module generated by jsii-pacmak 1.132.0 sets `go 1.25` in
local.go.mod. Go 1.22 + GOTOOLCHAIN=local refused to satisfy that
constraint. Switching to `go-version: stable` always installs the latest
stable release; the action itself remains SHA-pinned.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix link
Signed-off-by: ms280690 <mehul@sparkgeo.com>
* fix(upload): address code and security review findings
- Credential provider function replaces static creds snapshot — S3Client
now refreshes STS tokens automatically before expiry, preventing
ExpiredTokenException on long-running uploads
- _matchGlob rewritten as single-pass char loop — eliminates space-as-
sentinel bug, fixes ** depth-zero/depth-N and leading **/ glob patterns
- _walk now guards !entry.isSymbolicLink() — symlinks inside inputDir
are no longer followed, preventing accidental exfiltration
- Empty inputDir guard added — path.resolve('') silently returned CWD
- HeadObject error check now also accepts name === 'NoSuchKey' alongside
NotFound and httpStatusCode 404 for S3-compatible store correctness
- types.ts JSDoc corrected — removed false promise that .. sequences were
rejected; replaced with accurate description of symlink behaviour
- engines.node bumped to >=20 to match @aws-sdk/lib-storage transitive
constraint (was claiming >=18)
- Tests: +4 (empty dir, ** glob, NoSuchKey 404, non-404 error propagation)
Total: 64 tests pass, jsii build clean (Errors: 0 | Warnings: 0)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix security bugs and add tests
Signed-off-by: ms280690 <mehul@sparkgeo.com>
---------
Signed-off-by: ms280690 <mehul@sparkgeo.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
…targets (#49) * feat(phase-5.5): Docker smoke test infrastructure for all 5 language targets - Add smoke-tests/ with Docker Compose setup running JS, Python, Go, .NET, and Java smoke tests in isolated containers — no local toolchain required - Each non-JS container copies the node binary from node:22-slim (jsii runtimes spawn a Node subprocess at runtime); avoids apt/apk DNS failures during build - network_mode: host on all services — Docker bridge has no internet routing on the dev host; containers inherit host network stack for package managers - Add prescient-sdk-ts/justfile with recipes for build, test, package, and all Docker smoke test targets - Add prescient-sdk-ts/pnpm-workspace.yaml with nodeLinker: hoisted so pnpm places all transitive deps at root node_modules/, enabling jsii-pacmak to bundle ~54 packages (vs ~8 without hoisting) - Consolidate .gitignore into repo root; remove child gitignores from prescient-sdk-ts/ and smoke-tests/ - Update JSII_MIGRATION_PLAN.md: mark phases 1–6 done, add Phase 5.5 section, correct npm → pnpm, Node 20 → 22, remove googleClientSecret from public API docs, update Phase 7 CI yaml for pnpm Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(smoke-tests): address 6 review findings from PR #49 - js: node:22-alpine → node:22-slim (glibc/musl consistency with language containers) - python: *.whl glob instead of hardcoded version; pip cache volume - go: remove go mod tidy (go.sum committed); add go build cache; source mount ro - dotnet: named volumes for obj/ and bin/ so source mount can be ro - java: named volume for target/ so source mount can be ro; mvn -q → --no-transfer-progress Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(ci): add publish-sdk workflow for 5-registry jsii release
Triggers on vX.Y.Z tags. Builds once, uploads targets/ artifact, then
5 parallel publish jobs consume it:
- npm → registry.npmjs.org (secret: NPM_TOKEN)
- PyPI → pypi.org (secret: PYPI_JSII_API_TOKEN)
- NuGet → nuget.org (secret: NUGET_API_KEY)
- Maven → OSSRH/Central (secrets: MAVEN_USERNAME/PASSWORD/GPG_*)
- Go → sparkgeo/prescient-sdk-go (secret: GO_DEPLOY_TOKEN)
All ${{ }} expressions in run: steps moved to env: to prevent injection.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: update uv.lock (prune s390x greenlet wheels)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ci): address 7 review findings in publish-sdk workflow
1. publish-npm: publish pre-built targets/js/*.tgz directly (dist/ was
absent from artifact; jsii-pacmak packs dist/ into the tgz already)
2. publish-maven: replace broken `mvn deploy -f targets/java/pom.xml`
with publib-maven, which handles jsii local-repo layout, injects
distributionManagement, GPG-signs, and promotes OSSRH staging
3. Same as #2 — pom had no distributionManagement or maven-gpg-plugin
4. NuGet: move --api-key out of run: command string into env: block
5. Pin upload-artifact and download-artifact to SHA (was @v4 tag)
6. publish-go: check remote tag existence before git-tag to allow reruns
7. PyPI: upload targets/python/* (wheel + sdist) not just *.whl
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Using Claude code to use jsii https://github.com/aws/jsii and rewrite the sdk