fix: correct integration test failures and improve test reliability - #172
Open
jansdhillon wants to merge 4 commits into
Open
fix: correct integration test failures and improve test reliability#172jansdhillon wants to merge 4 commits into
jansdhillon wants to merge 4 commits into
Conversation
- Replace non-existent external_grpc_port/service assertions with correct port and hosts checks in grpc haproxy-route-tcp tests - Fix _legacy_haproxy_host to skip when website relation not present - Add verify=False to http request in test_legacy_haproxy_metrics_forbidden - Add bundle fixture dependency to test_modern_database_relation and test_legacy_db_relation so they wait for fresh model deploy - Fix test_non_leader_unit_redirects_leader_only_services to use haproxy IP with correct Host header instead of unresolvable root_url hostname - Use python -m pytest in Makefile to avoid stale venv shebang paths - Update integration bundle.yaml to use Charmhub refs and 2.8/stable haproxy - Update bundle-examples to use 2.8/stable haproxy channel Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…timeout to 30m - test_action_migrate_schema_fails_while_running was catching scenario.ActionFailed but jubilant raises jubilant.TaskError for failed actions - Increase WAIT_TIMEOUT_SECONDS from 20m to 30m; fresh deploy takes ~20m and the previous 5-success check was expiring just at the boundary Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR aims to get the integration test suite passing reliably again by aligning assertions with current HAProxy relation schemas, fixing a few test-environment assumptions (TLS/self-signed certs, hostnames), and updating bundles/tooling to match expected deployment behavior for Landscape Server and related charms.
Changes:
- Fix several integration tests to reflect current relation data schemas and runtime behavior (notably
haproxy-route-tcp, legacy HAProxy redirects, migrate-schema failure type). - Improve integration test reliability by ensuring fresh model deploy dependencies and increasing the deploy wait timeout.
- Update bundles/examples and test tooling (Charmhub refs/channels, HAProxy channel,
python -m pytestinvocation).
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
tests/integration/test_bundle.py |
Fixes multiple integration test failures (relation schema assertions, redirect host header, exception type, legacy HAProxy handling). |
tests/integration/conftest.py |
Increases integration wait timeout to better match real deploy times. |
tests/integration/bundle.yaml |
Updates integration bundle charm sources/channels and HAProxy channel to stabilize deployments. |
terraform/product/modules/landscape-scalable/terraform.tfvars.modern |
Aligns HAProxy channel with updated bundle expectations (2.8/stable). |
Makefile |
Switches integration test invocation to python -m pytest for more robust environment execution. |
bundle-examples/saas.bundle.yaml |
Moves HAProxy channel to 2.8/stable. |
bundle-examples/postgres14.bundle.yaml |
Updates HAProxy channel and normalizes boolean YAML values. |
bundle-examples/pgbouncer.bundle.yaml |
Moves HAProxy channel to 2.8/stable. |
bundle-examples/bundle.yaml |
Updates HAProxy channel but also substantially changes the example bundle composition/relations (scope appears broader than PR description). |
Suppressed comments (1)
tests/integration/test_bundle.py:693
- Same flakiness risk as above:
app_statusis read beforejuju.wait(...), but the unit address is used after. Read the unit address from a freshjuju.status()so the value is up to date.
unit_address = app_status.units[unit].public_address
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
593
to
596
| host = list(haproxy.units.values())[0].public_address | ||
| hostname = urlparse( | ||
| juju.config("landscape-server").get("root_url", "https://landscape.local/") | ||
| ).hostname |
| # The TCP haproxy-route schema has no "service" field to identify the backend, | ||
| # so verify the unit's own address is published as a backend host instead. | ||
| hosts = json.loads(data.get("hosts", "[]")) | ||
| unit_address = app_status.units[unit].public_address |
Comment on lines
24
to
43
| options: | ||
| landscape_ppa: "ppa:landscape/self-hosted-beta" | ||
| min_install: True | ||
| enable_hostagent_messenger: True | ||
| enable_ubuntu_installer_attach: True | ||
| min_install: false | ||
| enable_hostagent_messenger: true | ||
| enable_ubuntu_installer_attach: true | ||
| root_url: https://landscape.local/ | ||
| demo_data: True | ||
| demo_data: true | ||
| base: ubuntu@24.04 | ||
| landscape-debarchive: | ||
| charm: ch:landscape-debarchive | ||
| channel: edge | ||
| num_units: 1 | ||
| base: ubuntu@24.04 | ||
| landscape-task-handler: | ||
| charm: ch:landscape-task-handler | ||
| channel: edge | ||
| num_units: 1 | ||
| options: | ||
| task-handler-snap-channel: latest/edge | ||
| base: ubuntu@24.04 |
- Fetch fresh status after juju.wait() in grpc haproxy-route tests to avoid stale public_address when asserting unit IP in hosts - Guard root_url with 'or' fallback and hostname with 'or' fallback to prevent InvalidHeader/TypeError if config is None or schemeless - Revert unintended debarchive/task-handler additions from bundle-examples/bundle.yaml; keep only haproxy 2.8/edge -> 2.8/stable Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Manual testing
Deploy the regular (modern) bundle:
Then, run the integration tests against it (will skip destructive tests):
Deploy the legacy bundle to a different model and then switch to it:
Run the integration tests:
Then, run the integration tests using the model created by the tests to test the destructive tests:
(will take a long time)
Summary
Fixes several integration test failures that were preventing the test suite from passing.
Fixes
grpc haproxy-route-tcp tests:
external_grpc_port/servicekeys which don't exist in thehaproxy-route-tcpdatabag schemaportand verify the unit IP appears inhostsLegacy haproxy tests:
verify=Falseon http request (redirects to https with self-signed cert)_legacy_haproxy_hostto skip whenwebsiterelation isn't presentDB relation tests:
bundle: Nonefixture dependency so they wait for fresh model deployNon-leader redirect test:
root_urlhostname; fixed to use haproxy IP with correctHostheadermigrate-schema failure test:
scenario.ActionFailed(ops-scenario type) but jubilant raisesjubilant.TaskErrorDeploy timeout:
WAIT_TIMEOUT_SECONDSfrom 20m to 30m — fresh deploy takes ~20m and old value was too tightMakefile:
pytest→python -m pytestto avoid broken shebang paths across environmentsBundle updates:
tests/integration/bundle.yaml: Charmhub refs,2.8/stablehaproxy, removedredirect_https: nonebundle-examples/: haproxy channel2.8/edge→2.8/stableTesting
landscape-server-operator-build(live): 14 passed, 26 skipped ✅legacy(live): 10 passed, 30 skipped ✅