Skip to content

QA infra test improvements #417

Description

@MichaelThamm

For context, first read this HowTo:

We should improve our QA tests so we have a maintainable implementation and better coverage. This can be done in a few ways:

  • Add pytests to our sqa folder under tests, and any other helper functions there. These are executed by the sqa_tests in the root repo.
  • We should try to avoid having a test variant per product flavor: COS / COS Lite. The main differences are otelcol and timeouts
    • We likely don't want to wait 2h for active/idle for COS Lite, although this is valid for COS.
    • Consider a good design for checking which flavor we have and piping that into each helper script.
  • We should choose which tests we already have in tests/solution and port them over to tests/sqa, or better yet, make them reusable?
  • We should output our results in xUnit format, like how the Juju team does it. They do it in GoLang, but we can do it in some Pythonic way.
  • We could also address this warning by generating a metrics.json file (likely a stretch goal).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions