diff --git a/pyproject.toml b/pyproject.toml index 72cdf9c..42152db 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -34,6 +34,8 @@ test = [ "pytest>=8.0", "pytest-asyncio>=0.24", "aioresponses>=0.7", + # aioresponses 0.7.8 is not compatible with aiohttp 3.14's ClientResponse constructor. + "aiohttp<3.14", "pytest-cov>=6.0", ] @@ -43,4 +45,4 @@ asyncio_mode = "auto" [project.urls] Homepage = "https://github.com/Autohive-AI/integrations-sdk" -Issues = "https://github.com/Autohive-AI/integrations-sdk/issues" \ No newline at end of file +Issues = "https://github.com/Autohive-AI/integrations-sdk/issues" diff --git a/skills/building-integration/SKILL.md b/skills/building-integration/SKILL.md index 4c7fbe2..cb1720a 100644 --- a/skills/building-integration/SKILL.md +++ b/skills/building-integration/SKILL.md @@ -145,7 +145,16 @@ Only runs tests in `test_*_unit.py` files (configured in `pyproject.toml`). If the integration has `test_*_integration.py` files, check whether they reference environment variables via `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers. Every referenced variable must appear as a blank template entry in the repository root `.env.example`. -This is a manual review/build requirement: `.env` stays local and uncommitted, but `.env.example` is the committed contract that tells humans, reviewers, and automation what to provide. +This is a manual review/build requirement: `.env` stays local and uncommitted, but `.env.example` is the committed contract that tells humans, reviewers, and automation what to provide. Missing entries should be treated as a validation failure for PRs adding or changing integration tests. + +Use a focused cross-check: + +```bash +grep -RInE 'env_credentials\(|os\.environ|getenv\(' /tests/test_*_integration.py +grep -nE 'INTEGRATION_PREFIX|SERVICE_PREFIX' .env.example +``` + +Document required credentials, optional test IDs, and destructive-test-only variables. If an env var read is unused, remove it rather than documenting unnecessary setup. ### Step 6 β€” Auto-fix lint and formatting issues @@ -201,4 +210,4 @@ Key settings: | Config-code sync | Ensure actions in `config.json` match handlers in source code | | Missing test files | Create `tests/test__unit.py` (see `writing-unit-tests` skill) | | Import errors | Check `requirements.txt` has all dependencies; check `__init__.py` exports | -| Integration tests use env vars but `.env.example` missing entries | Add blank entries for every referenced env var to root `.env.example` | +| Integration tests use env vars but `.env.example` missing entries | Add blank entries for every referenced env var to root `.env.example`; treat as blocking for PRs adding/changing integration tests | diff --git a/skills/reviewing-integration-prs/SKILL.md b/skills/reviewing-integration-prs/SKILL.md index a34292d..f1af3f5 100644 --- a/skills/reviewing-integration-prs/SKILL.md +++ b/skills/reviewing-integration-prs/SKILL.md @@ -14,6 +14,7 @@ The goal is not just a generic code review. Validate the PR against the relevant - Review the actual diff and the surrounding integration code; do not rely only on CI status. - Treat security and secret leakage as blockers. - Treat failed tooling/validation as blockers unless there is a documented, intentional exception. +- Treat missing root `.env.example` entries as blockers when a PR adds or changes integration tests that read env vars. - Use the existing skills as sub-skill checklists when their context applies. - Prefer actionable review comments with file/line references and concrete fixes. - Separate blockers from should-fix items and minor suggestions. @@ -104,6 +105,8 @@ For migrations to public, apply the `migrating-private-integration` security/saf ### 5. Tests and `.env.example` +The root `.env.example` check is mandatory for any PR that adds or changes `test_*_integration.py`. Do not rely on CI to catch this; review it manually against the test file. + For unit tests, apply `writing-unit-tests`: - Tests are named `test_*_unit.py`. @@ -118,8 +121,16 @@ For integration tests, apply `writing-integration-tests`: - `live_context` returns `FetchResponse` for SDK 2.x `context.fetch` paths. - Tests skip cleanly when credentials or required test IDs are missing. - Every env var read by `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers is listed as a blank template entry in root `.env.example`. +- Optional test IDs, destructive-test-only variables, and module-level env constants count; either document them in root `.env.example` or remove unused reads. + +Use a focused search while reviewing: + +```bash +grep -RInE 'env_credentials\(|os\.environ|getenv\(' /tests/test_*_integration.py +grep -nE 'INTEGRATION_PREFIX|SERVICE_PREFIX' .env.example +``` -Missing `.env.example` entries are a should-fix at minimum, and often a blocker for PRs whose main purpose is adding/changing integration tests. +Missing `.env.example` entries are blockers when the PR adds or changes integration tests that read env vars. If the PR only exposes a pre-existing omission outside the changed tests, call it a should-fix unless it blocks the changed behavior. ### 6. SDK v2 upgrade review @@ -147,8 +158,8 @@ Check: Use these categories: -- **🚫 Blocker** β€” must fix before merge. Security leaks, broken validation, schema/action mismatch, new integration input drift, missing required files, broken imports/tests, unsafe public migration. -- **⚠️ Should-fix** β€” strongly recommended before merge. Missing `.env.example` entries, incomplete test coverage for changed behavior, unclear docs, overly broad error handling, avoidable warnings. +- **🚫 Blocker** β€” must fix before merge. Security leaks, broken validation, schema/action mismatch, new integration input drift, missing required files, broken imports/tests, unsafe public migration, or missing root `.env.example` entries for env vars read by newly added/changed integration tests. +- **⚠️ Should-fix** β€” strongly recommended before merge. Pre-existing `.env.example` omissions not introduced by the PR, incomplete test coverage for changed behavior, unclear docs, overly broad error handling, avoidable warnings. - **πŸ’‘ Suggestion** β€” optional improvements. Naming clarity, small refactors, additional examples, extra edge-case tests. ## Review Output Template @@ -184,5 +195,5 @@ Before completing the review, verify: - [ ] Tooling/CI status was checked or local validation was run. - [ ] Config/code/schema sync was inspected. - [ ] Unit/integration test changes were reviewed against the appropriate test skill. -- [ ] Root `.env.example` was checked when integration tests reference env vars. +- [ ] Root `.env.example` was checked when integration tests reference env vars; missing entries introduced by the PR were treated as blockers. - [ ] Findings are actionable and severity-labelled. diff --git a/skills/upgrading-sdk-v2/SKILL.md b/skills/upgrading-sdk-v2/SKILL.md index 46ed38b..8989f04 100644 --- a/skills/upgrading-sdk-v2/SKILL.md +++ b/skills/upgrading-sdk-v2/SKILL.md @@ -210,7 +210,7 @@ async def real_fetch(url, *, method="GET", json=None, headers=None, **kwargs): ) ``` -If the integration tests read credentials or test resource IDs from environment variables, also verify the repository root `.env.example` lists every variable used by those tests. Add missing entries in the same PR. This applies to variables read through `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers. The local `.env` file remains uncommitted; `.env.example` is the committed setup contract for humans and tooling. +If the integration tests read credentials or test resource IDs from environment variables, also verify the repository root `.env.example` lists every variable used by those tests. Add missing entries in the same PR. This applies to variables read through `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers. Required credentials, optional test IDs, and destructive-test-only variables all count. The local `.env` file remains uncommitted; `.env.example` is the committed setup contract for humans and tooling. Missing entries should block SDK-upgrade PRs that add or change integration tests. ### Step 7 β€” Local validation (required before pushing) @@ -265,7 +265,7 @@ Before considering an integration upgraded, verify: - [ ] Unit test error assertions use `result.type == ResultType.ACTION_ERROR` and `result.result.message` - [ ] `pytest.raises(ValidationError)` replaced with `result.type == ResultType.VALIDATION_ERROR` - [ ] Integration test `real_fetch` returns `FetchResponse(...)` (if applicable) -- [ ] Integration test env vars are documented in root `.env.example` (if integration tests read env vars) +- [ ] Integration test env vars are documented in root `.env.example` (if integration tests read env vars, including optional and destructive-test-only variables) - [ ] `FetchResponse` and `ResultType` are imported where needed - [ ] All unit tests pass - [ ] `ruff check` and `ruff format --config ../autohive-integrations-tooling/ruff.toml` pass @@ -303,4 +303,4 @@ Before considering an integration upgraded, verify: 10. **PyPI package name collision**: If your integration folder name matches a PyPI package the source imports (e.g. an integration in `supadata/` that does `from supadata import Supadata`), an empty `/__init__.py` will shadow the real PyPI package and every test fails with `ImportError`. Drop `/__init__.py` β€” the validator's "missing __init__.py" warning is correct to ignore in this case, and the Lambda runtime is unaffected (the entry point is the action source file, not the package). See the `writing-unit-tests` skill for the matching test-side guidance. -11. **Missing `.env.example` entries after touching integration tests**: SDK upgrades often touch `live_context` and credential handling. If the integration test file references `FOO_ACCESS_TOKEN`, `FOO_TEST_ID`, or similar, root `.env.example` must contain `FOO_ACCESS_TOKEN=`, `FOO_TEST_ID=`, etc. Do not rely on future contributors or automation to grep test code for setup requirements. +11. **Missing `.env.example` entries after touching integration tests**: SDK upgrades often touch `live_context` and credential handling. If the integration test file references `FOO_ACCESS_TOKEN`, `FOO_TEST_ID`, or similar, root `.env.example` must contain `FOO_ACCESS_TOKEN=`, `FOO_TEST_ID=`, etc. Optional IDs and destructive-test-only variables count too. Do not rely on future contributors or automation to grep test code for setup requirements; missing entries should block PRs that add or change those tests. diff --git a/skills/writing-integration-tests/SKILL.md b/skills/writing-integration-tests/SKILL.md index 7d4fbb5..f1f66e0 100644 --- a/skills/writing-integration-tests/SKILL.md +++ b/skills/writing-integration-tests/SKILL.md @@ -5,6 +5,18 @@ description: "Writes pytest end-to-end integration tests for an Autohive integra # Writing Integration Tests for an Integration +## Required: root `.env.example` update + +If a PR adds or changes integration tests that read environment variables, it must update the repository root `.env.example` in the same PR. This is part of the integration-test deliverable, not optional documentation. + +This applies to every variable read through `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers, including: + +- Required credentials such as access tokens and API keys +- Optional test resource IDs used to avoid listβ†’get chaining +- Variables used only by destructive tests + +If a variable is read by the test file, either add a blank template entry to root `.env.example` or remove the unused env-var read. Missing entries should block integration-test PRs until fixed. + ## Prerequisites - The integration must be on **SDK 2.0.0** (`autohive-integrations-sdk~=2.0.0` in `requirements.txt`) @@ -134,7 +146,16 @@ Why this matters: - Automated and AI tooling can discover which credentials, test IDs, and optional knobs it needs to ask for. - Reviewers have one canonical place to verify the local test contract. -Add every variable used by `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers. Include both required credentials and optional test resource IDs. Keep real secrets only in local `.env`; never commit `.env`. +Add every variable used by `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers. Include required credentials, optional test resource IDs, and destructive-test-only variables. Keep real secrets only in local `.env`; never commit `.env`. + +Before finishing, run a focused cross-check: + +```bash +grep -RInE 'env_credentials\(|os\.environ|getenv\(' /tests/test_*_integration.py +grep -nE 'MYINTEGRATION|SERVICE_PREFIX' .env.example +``` + +Every env var found in the integration test file must be present as a blank template entry in root `.env.example`. If the variable is no longer used by tests, remove the env-var read instead of documenting unnecessary setup. Use a clearly labeled block in root `.env.example`: @@ -145,7 +166,7 @@ MYINTEGRATION_TEST_ITEM_ID= MYINTEGRATION_TEST_PROJECT_ID= ``` -Before finishing, cross-check the integration test file against `.env.example`: every env var referenced in tests must appear in `.env.example`, even if the test skips when it is missing. +Before finishing, cross-check the integration test file against `.env.example`: every env var referenced in tests must appear in `.env.example`, even if the test skips when it is missing or the variable is optional. ## The live_context Fixture @@ -466,7 +487,7 @@ Write actions (create/update/delete) need at least: 1. **Read the integration source** β€” understand each action and its auth mechanism 2. **Check `config.json`** β€” determine auth type (none / API key / platform OAuth) 3. **Choose the right `live_context` variant** β€” see the three variants above -4. **Document env vars in root `.env.example` (required)** β€” add every env var referenced by the integration test file in the same PR +4. **Document env vars in root `.env.example` (required)** β€” add every env var referenced by the integration test file in the same PR; missing entries should block integration-test PRs 5. **Write read-only tests first** β€” these are safe to run repeatedly 6. **Add destructive tests** with `@pytest.mark.destructive` for write actions 7. **Run read-only tests**: @@ -494,7 +515,7 @@ Write actions (create/update/delete) need at least: 5. **Destructive test cleanup**: Always clean up created data at the end of lifecycle tests. Use `os.getpid()` in created object names to avoid collisions when running tests in parallel. -6. **Missing `.env.example` entries**: If you add `env_credentials("FOO")`, `os.environ.get("FOO")`, or similar test env-var usage, add `FOO=` to root `.env.example` before you commit. Do not make reviewers or future contributors grep test files to discover setup requirements. +6. **Missing `.env.example` entries**: If you add `env_credentials("FOO")`, `os.environ.get("FOO")`, `os.getenv("FOO")`, `os.environ["FOO"]`, or similar test env-var usage, add `FOO=` to root `.env.example` before you commit. Do not make reviewers or future contributors grep test files to discover setup requirements. Optional IDs and destructive-test-only variables count too. 7. **OAuth token expiry**: Platform OAuth tokens expire. Document the refresh process in the integration's README or root `.env.example`.