feat: read solution data from attributes, stop writing legacy fields [3/3] by almeidaraul · Pull Request #850 · canonical/test_observer

almeidaraul · 2026-08-18T12:33:32Z

Part 3 of 3 (stacked). Base: TO-404/2-write-both (#849).

What

Migrate + read-new step: backfill existing data, switch reads to attributes, and stop writing the legacy solution-specific fields.

Backfill migration copies existing bundled_builds_hash / bundled build associations into attributes for rows written before the write-both code.
Controllers now read solution identity from attributes and stop writing the legacy fields. StartSolutionTestExecutionRequest accepts attributes (with track/source kept as deprecated legacy inputs folded into it); ArtefactPatch accepts attributes and drops bundled_builds. get_artefact_versions matches on family instead of the bundled-builds hash.
Remove the bundled_builds relationship/hash, association table and related ORM event hooks from the model, dropping them from API responses, repository and data generator.

Safety

The legacy DB column and table are left in place (dropped later by a separate destructive/contract migration); alembic check ignores them via transitional include_object exclusions. Safe to deploy after the write-both release during a rolling upgrade.

Not included (future PR)

The destructive contract migration (drop bundled_builds_hash + artefact_bundled_builds_association, swap unique_solution to (name, version)) is a separate 4th PR deployed after this one. It stacks on this branch with only a down_revision bump to d315c1f212b9.

Review note

Review only against the base branch (#849). Merge/deploy after #849.

Third step of the expand/contract migration to a generic `attributes` field. - Backfill migration copies existing bundled_builds_hash / bundled build associations into `attributes` for rows written before the write-both code. - Controllers now read solution identity from `attributes` and stop writing the legacy solution-specific fields. `StartSolutionTestExecutionRequest` accepts `attributes` (with track/source kept as deprecated legacy inputs folded into it); `ArtefactPatch` accepts `attributes` and drops bundled_builds. get_artefact_versions matches on family instead of the bundled builds hash. - Remove the bundled_builds relationship/hash, association table and related ORM event hooks from the model, dropping them from the API responses, repository and data generator. The legacy DB column and table are left in place (dropped later by the destructive migration); `alembic check` ignores them via transitional include_object exclusions. The old columns/table remain in the database, so this is safe to deploy after the write-both release during a rolling upgrade. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Contract step of the expand/contract rolling-upgrade sequence. After the backfill + read-new release (#850) is deployed, remove the now-unused legacy solution-specific fields: - Drop `artefact.bundled_builds_hash` column - Drop the `artefact_bundled_builds_association` table - Swap the `unique_solution` index to a simple (name, version) uniqueness - Reset the transitional `alembic check` expand/contract exclusions in env.py now that the ORM and DB schema agree again Migration `8bd1f5009f02` revises the backfill migration `d315c1f212b9`. Deploy only after #850 has merged and rolled out. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR completes the “read-new” step of migrating solution-related identity data from legacy solution-specific columns/relationships into the generic artefact.attributes JSONB field, including a backfill migration and API/model updates to stop writing the legacy fields.

Changes:

Add a data-only Alembic migration to backfill artefact.attributes from legacy bundled-build columns/associations.
Update controllers and request/patch models to read/write solution identity via attributes (keeping legacy track/source as deprecated inputs that are folded into attributes).
Remove the legacy bundled-build relationship/hash from ORM/models and update OpenAPI + tests accordingly.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
backend/migrations/env.py	Adds transitional `include_object` exclusions so `alembic check` ignores legacy solution schema during expand/contract rollout.
backend/migrations/versions/2026_08_17_1724-d315c1f212b9_backfill_artefact_attributes.py	Data-only backfill of bundled-build legacy data into `artefact.attributes`.
backend/schemata/openapi.json	Updates API schema to remove bundled-build fields and add/adjust `attributes` + deprecated solution inputs.
backend/test_observer/controllers/artefacts/artefacts.py	Updates PATCH handling for `attributes` and changes version-history matching (now includes `family`).
backend/test_observer/controllers/artefacts/builds.py	Stops eager-loading the removed `bundled_in` relationship.
backend/test_observer/controllers/artefacts/models.py	Removes bundled-build response fields and adds `attributes` to `ArtefactPatch`.
backend/test_observer/controllers/test_executions/models.py	Adds `attributes` to solution start request and folds deprecated `track`/`source` into it.
backend/test_observer/controllers/test_executions/start_test.py	Stops writing legacy solution fields; creates solution artefacts using `attributes`.
backend/test_observer/data_access/models.py	Removes bundled-build association/hash and updates the solution uniqueness index definition.
backend/test_observer/data_access/repository.py	Removes bundled-build eager-loading option from repository queries.
backend/tests/controllers/artefacts/test_artefacts.py	Updates artefact API tests for `attributes`, removal of bundled-builds, and family-safe version histories.
backend/tests/controllers/artefacts/test_builds.py	Updates build API tests after removal of `bundled_in`.
backend/tests/controllers/test_executions/test_reruns.py	Updates rerun payload expectations after removal of bundled-build fields.
backend/tests/controllers/test_executions/test_start_test.py	Updates solution start-test tests to use `attributes` and deprecated legacy input folding rules.
backend/tests/data_access/test_models.py	Adds model-level tests around the updated solution uniqueness expectations.
backend/tests/data_generator.py	Switches test artefact generation from `bundled_builds` to `attributes`.
backend/tests/migrations/test_d315c1f212b9_backfill_artefact_attributes.py	Adds migration tests validating the backfill behavior and non-destructive guarantees.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

            case StartSolutionTestExecutionRequest():
-                filter_kwargs["track"] = self.request.track
-                filter_kwargs["source"] = self.request.source
-                filter_kwargs["stage"] = self.request.execution_stage
-                filter_kwargs["bundled_builds_hash"] = calculate_bundled_builds_hash([])
-                # Write-both (expand/contract): keep populating the legacy solution
-                # fields above while also mirroring the identity into the new
-                # ``attributes`` field, so readers can switch to it in a later release.
-                creation_kwargs = {
-                    "attributes": {
-                        "track": self.request.track,
-                        "source": self.request.source,
-                    }
-                }
+                creation_kwargs["attributes"] = self.request.attributes

        self.artefact = get_or_create(self.db, Artefact, filter_kwargs=filter_kwargs, creation_kwargs=creation_kwargs)


    return db.scalars(
        select(Artefact)
        .where(Artefact.name == artefact.name)
+        .where(Artefact.family == artefact.family)
        .where(Artefact.track == artefact.track)
        .where(Artefact.branch == artefact.branch)
        .where(Artefact.series == artefact.series)
        .where(Artefact.repo == artefact.repo)
        .where(Artefact.os == artefact.os)
        .where(Artefact.release == artefact.release)
        .where(Artefact.source == artefact.source)
-        .where(Artefact.bundled_builds_hash == artefact.bundled_builds_hash)
-        .options(selectinload(Artefact.bundled_builds))
        .order_by(Artefact.id.desc())


almeidaraul mentioned this pull request Aug 18, 2026

feat: drop solution-specific bundled build fields (contract) [4/4] #851

Open

almeidaraul force-pushed the TO-404/3-migrate-read-new branch from 6722c4a to e2c418c Compare August 18, 2026 13:13

fix: linting

3c2ede9

almeidaraul marked this pull request as ready for review August 18, 2026 14:25

Copilot AI lite review requested due to automatic review settings August 18, 2026 14:25

Copilot started reviewing on behalf of almeidaraul August 18, 2026 14:25 View session

Copilot AI reviewed Aug 18, 2026

View reviewed changes

Navigation Menu

Uh oh!

feat: read solution data from attributes, stop writing legacy fields [3/3] - #850

feat: read solution data from attributes, stop writing legacy fields [3/3]#850
almeidaraul wants to merge 2 commits into
TO-404/2-write-bothfrom
TO-404/3-migrate-read-new

almeidaraul commented Aug 18, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

almeidaraul commented Aug 18, 2026

What

Safety

Not included (future PR)

Review note

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants