test: add live integration tests for guard, connect, and registration#44
Open
beonde wants to merge 2 commits into
Open
test: add live integration tests for guard, connect, and registration#44beonde wants to merge 2 commits into
beonde wants to merge 2 commits into
Conversation
Add tests/integration/ with 18 tests covering: - test_guard_live.py: evaluate_tool_access against live gRPC (7 tests) - test_connect_live.py: MCPServerIdentity.connect() full flow (5 tests) - test_registration_live.py: keygen + registration round-trip (6 tests) All tests skip gracefully when infrastructure is unavailable. Markers: @pytest.mark.integration + skipif for missing core/server. Also adds: - docker-compose.test.yml for local testing with postgres + server - .github/workflows/live-integration-tests.yml CI workflow - pyproject.toml: register 'integration' marker
There was a problem hiding this comment.
Pull request overview
Adds a live integration-test suite that exercises capiscio-mcp-python end-to-end against real capiscio-core (gRPC) and capiscio-server (REST), plus local/CI infrastructure to run those tests.
Changes:
- Introduces new
tests/integration/live tests for guard evaluation, server identity connect, and registration flows. - Adds integration-test fixtures and registers a new
integrationpytest marker. - Adds local Docker Compose setup and a GitHub Actions workflow to run the live integration tests in CI.
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/integration/test_registration_live.py | Adds live registration tests (keygen/register/setup) against real infrastructure. |
| tests/integration/test_guard_live.py | Adds live tests for evaluate_tool_access()/guard behavior against a running core. |
| tests/integration/test_connect_live.py | Adds live tests for MCPServerIdentity.connect() end-to-end flow. |
| tests/integration/conftest.py | Provides integration fixtures (server URL, API key) and resets the CoreClient between tests. |
| tests/integration/init.py | Initializes the integration test package. |
| pyproject.toml | Registers the integration pytest marker. |
| docker-compose.test.yml | Adds local infra (Postgres + capiscio-server + test runner) for running live tests. |
| .github/workflows/live-integration-tests.yml | Adds CI workflow to build core/server and run the live integration tests. |
Comments suppressed due to low confidence (3)
tests/integration/test_registration_live.py:66
- This uniqueness check is currently looking up
public_key, but the registration API returnspublic_key_pem. As written,key1/key2will beNoneand the assertion will fail (or give a misleading result).
key1 = result1.get("public_key") or getattr(result1, "public_key_pem", None)
key2 = result2.get("public_key") or getattr(result2, "public_key_pem", None)
assert key1 != key2
tests/integration/test_connect_live.py:90
MCPServerIdentity.close()is synchronous; awaiting it here will raiseTypeError. Useidentity.close()(noawait).
assert identity.did is not None
await identity.close()
except CoreConnectionError:
pytest.skip("Core connection failed")
tests/integration/test_connect_live.py:119
close()is synchronous onMCPServerIdentity; theseawaitcalls will raiseTypeErrorand can break the idempotency test.
)
did1 = identity1.did
await identity1.close()
identity2 = await MCPServerIdentity.connect(
server_id="test-integration-idempotent",
api_key=api_key,
server_url=server_url,
keys_dir=keys_dir,
auto_badge=False,
)
did2 = identity2.did
await identity2.close()
Comment on lines
+49
to
+55
| result = await generate_server_keypair() | ||
| except Exception as e: | ||
| pytest.skip(f"Core keygen not available: {e}") | ||
|
|
||
| assert "private_key" in result or hasattr(result, "private_key_pem") | ||
| assert "public_key" in result or hasattr(result, "public_key_pem") | ||
|
|
Comment on lines
+75
to
+85
| server_id = f"test-reg-{uuid.uuid4().hex[:8]}" | ||
| test_did = f"did:key:z6Mk{uuid.uuid4().hex[:32]}" | ||
| test_pubkey = "MCowBQYDK2VwAyEA" + "A" * 43 + "=" | ||
|
|
||
| try: | ||
| result = await register_server_identity( | ||
| server_id=server_id, | ||
| api_key=api_key, | ||
| did=test_did, | ||
| public_key=test_pubkey, | ||
| ca_url=server_url, |
Comment on lines
+94
to
+120
| async def test_register_idempotent(self, server_url, api_key): | ||
| """Registering the same server_id twice should be idempotent (409 = OK).""" | ||
| server_id = f"test-idempotent-{uuid.uuid4().hex[:8]}" | ||
| test_did = f"did:key:z6Mk{uuid.uuid4().hex[:32]}" | ||
| test_pubkey = "MCowBQYDK2VwAyEA" + "B" * 43 + "=" | ||
|
|
||
| try: | ||
| await register_server_identity( | ||
| server_id=server_id, | ||
| api_key=api_key, | ||
| did=test_did, | ||
| public_key=test_pubkey, | ||
| ca_url=server_url, | ||
| ) | ||
| result = await register_server_identity( | ||
| server_id=server_id, | ||
| api_key=api_key, | ||
| did=test_did, | ||
| public_key=test_pubkey, | ||
| ca_url=server_url, | ||
| ) | ||
| except Exception as e: | ||
| if "401" in str(e) or "403" in str(e): | ||
| pytest.skip(f"Auth error: {e}") | ||
| if "409" not in str(e): | ||
| raise | ||
|
|
Comment on lines
+146
to
+156
| try: | ||
| result = await setup_server_identity( | ||
| server_id=server_id, | ||
| api_key=api_key, | ||
| ca_url=server_url, | ||
| ) | ||
| except Exception as e: | ||
| if "401" in str(e) or "403" in str(e): | ||
| pytest.skip(f"Auth error: {e}") | ||
| pytest.skip(f"Setup not available: {e}") | ||
|
|
Comment on lines
+61
to
+66
| assert os.path.exists(os.path.join(keys_dir, "did.txt")) | ||
| assert identity.did is not None | ||
| assert identity.did.startswith("did:") | ||
|
|
||
| await identity.close() | ||
| except CoreConnectionError: |
Comment on lines
+128
to
+142
| async def test_connect_context_manager(self, server_url, api_key): | ||
| """MCPServerIdentity should work as an async context manager.""" | ||
| with tempfile.TemporaryDirectory() as keys_dir: | ||
| try: | ||
| async with MCPServerIdentity.connect( | ||
| server_id="test-integration-ctx", | ||
| api_key=api_key, | ||
| server_url=server_url, | ||
| keys_dir=keys_dir, | ||
| auto_badge=False, | ||
| ) as identity: | ||
| assert identity.did is not None | ||
| except (CoreConnectionError, TypeError): | ||
| pytest.skip("Context manager or core not available") | ||
| except Exception as e: |
Comment on lines
+15
to
+26
| _core_available = bool( | ||
| os.environ.get("CAPISCIO_CORE_ADDR") | ||
| or os.environ.get("CAPISCIO_BINARY_PATH") | ||
| ) | ||
|
|
||
| pytestmark = [ | ||
| pytest.mark.integration, | ||
| pytest.mark.skipif( | ||
| not _core_available, | ||
| reason="CAPISCIO_CORE_ADDR or CAPISCIO_BINARY_PATH not set", | ||
| ), | ||
| ] |
Comment on lines
+49
to
+60
| environment: | ||
| - CAPISCIO_SERVER_URL=http://server:8080 | ||
| - CAPISCIO_CORE_ADDR= # empty = embedded mode (auto-download binary) | ||
| - CAPISCIO_API_KEY=test-integration-key | ||
| - PYTHONPATH=/workspace | ||
| - PYTEST_ARGS=${PYTEST_ARGS:--v} | ||
| depends_on: | ||
| server: | ||
| condition: service_healthy | ||
| command: > | ||
| pytest tests/integration/ -v --tb=short | ||
| -m integration ${PYTEST_ARGS} |
Comment on lines
+130
to
+138
| - name: Run live integration tests | ||
| env: | ||
| CAPISCIO_SERVER_URL: http://localhost:8081 | ||
| CAPISCIO_CORE_ADDR: localhost:50051 | ||
| CAPISCIO_API_KEY: test-integration-key | ||
| run: | | ||
| pytest tests/integration/ -v --tb=short -m integration \ | ||
| --junitxml=integration-results.xml || true | ||
|
|
Comment on lines
+118
to
+125
| - name: Start capiscio-core gRPC | ||
| run: | | ||
| export CAPISCIO_BINARY_PATH="${{ github.workspace }}/_capiscio-core/bin/capiscio" | ||
| $CAPISCIO_BINARY_PATH serve --port 50051 & | ||
| CORE_PID=$! | ||
| echo "CORE_PID=$CORE_PID" >> $GITHUB_ENV | ||
| echo "CAPISCIO_CORE_ADDR=localhost:50051" >> $GITHUB_ENV | ||
|
|
- Fix hasattr() on dict → 'key in dict' (test_registration_live.py) - Fix await on sync close() → direct close() (test_connect_live.py) - Fix async with on sync context manager → await connect then sync with - Add assertions to idempotent registration test - Narrow broad 'except Exception' skip to connection-specific errors - Fix gRPC binary command: 'serve' → 'rpc --address' (live-integration-tests.yml) - Remove '|| true' from pytest so test failures gate the workflow
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add live integration tests for capiscio-mcp-python that test against real capiscio-core (gRPC) and capiscio-server (REST API) infrastructure.
Previously, all 386 tests used mocked gRPC. These 18 new tests exercise the real Python → gRPC → Go core → REST API paths.
Test Files
test_guard_live.pyevaluate_tool_access()with anonymous/API key credentials, params hashing, trusted issuers, allowed tools, decision cachetest_connect_live.pyMCPServerIdentity.connect()keypair generation, full flow (badge issuance), idempotent registration, context manager, invalid key rejectiontest_registration_live.pygenerate_server_keypair(),register_server_identity(),setup_server_identity()end-to-endInfrastructure
@pytest.mark.skipif)docker-compose.test.yml— PostgreSQL + capiscio-server for local testing.github/workflows/live-integration-tests.yml— CI workflow: builds core binary, starts server + postgres, runs integration testspyproject.toml— registersintegrationmarkerVerification
Existing test suite unaffected.