test: refactor automation strategy around real APISIX E2E and Ginkgo

[guoqqqi]

Background

The a6 test strategy has now been clarified and implemented in phases.

The current repository policy is:

1. Do not add new command-layer unit tests that mock the APISIX Admin API.
2. Command behavior that depends on APISIX should be covered by E2E against a real APISIX Admin API.
3. Existing mock Admin API unit tests are temporarily retained. Their removal is intentionally deferred and should be decided in a separate cleanup phase.
4. E2E scenarios should be written with Ginkgo/Gomega.
5. New E2E coverage should primarily exercise the CLI lifecycle: create, update, delete, list/export, and especially get assertions through the CLI.
6. E2E should not require validating the real data-plane/proxy chain by default. Gateway/proxy traffic checks should only be added when the feature specifically requires that behavior.

Goal

Refactor the a6 automated test strategy so that:

• unit tests are reserved for pure local logic and self-contained helpers
• command behavior coverage is centered on real APISIX E2E
• Ginkgo/Gomega is the standard framework for E2E organization and execution
• CI remains stable during the migration
• legacy mock Admin API tests are not removed in this phase

Completed Work

• docs(test): redefine unit-vs-e2e policy for a6 #20 documented the new unit-vs-e2e testing policy
• test(e2e): bootstrap Ginkgo/Gomega framework for real APISIX scenarios #23 bootstrapped the Ginkgo/Gomega E2E framework
• test: migrate core command coverage from mocked Admin API tests to real E2E #22 migrated the first core-resource command batch to Ginkgo E2E
• test: migrate remaining resource commands and remove legacy mock-command coverage #21 migrated the remaining command families to Ginkgo E2E under the current boundary

Related merged PRs include:

• test: migrate route service upstream coverage to ginkgo e2e #24 route/service/upstream Ginkgo E2E
• test: migrate config and group commands to ginkgo e2e #25 config and group command Ginkgo E2E
• test: assert e2e resources through cli reads #26 CLI read assertions for E2E resources
• test: add consumer credential ginkgo e2e #27 consumer/credential Ginkgo E2E
• test: add proto secret ginkgo e2e #28 proto/secret Ginkgo E2E
• test: add ssl stream route ginkgo e2e #29 ssl/stream-route Ginkgo E2E
• test: add utility command ginkgo e2e #30 utility command Ginkgo E2E
• test: add debug command ginkgo e2e #31 debug command Ginkgo E2E

Final Status

The strategy migration is complete under the current boundary.

Future work, if needed, should be tracked separately:

• decide whether and when to remove legacy mock Admin API unit tests
• add more scenario coverage for specific gaps
• add proxy/data-plane validation only for features that explicitly require it

[guoqqqi]

Implementation plan for the new testing strategy:

1. Policy/docs phase

• Track in docs(test): redefine unit-vs-e2e policy for a6 #20
• Rewrite docs so the repository no longer recommends mocked APISIX Admin API tests for command behavior
• Define the allowed unit-test boundary: pure logic, self-contained helpers, and no external-environment mocking
• Document Ginkgo/Gomega as the standard E2E direction

2. Ginkgo E2E bootstrap phase

• Track in test(e2e): bootstrap Ginkgo/Gomega framework for real APISIX scenarios #23
• Add Ginkgo/Gomega and establish a reusable suite/framework under test/e2e
• Keep the current real APISIX/etcd/httpbin topology and environment variables
• Introduce shared helpers for binary setup, admin API access, gateway checks, Eventually-style waits, cleanup, and capability-gated skips

3. First migration batch

• Track in test: migrate core command coverage from mocked Admin API tests to real E2E #22
• Move the highest-value command coverage from mocked Admin API tests to real APISIX E2E for route, service, upstream, plugin-config, and global-rule/label/export combinations
• Ensure each migrated family has CRUD plus at least one real gateway traffic validation path
• Remove or retire the corresponding mocked command tests once equivalent E2E coverage is in place

4. Remaining migration and cleanup

• Track in test: migrate remaining resource commands and remove legacy mock-command coverage #21
• Continue with consumer, credential, consumer-group, ssl, stream-route, proto, secret, plugin-metadata, and the remaining command families where behavior still depends on mocked Admin API responses
• Preserve only pure-logic unit tests outside E2E
• Bring docs and CI to the final steady-state once the migration is complete

Execution order should be: #20 -> #23 -> #22 -> #21. The intent is to freeze the new rules first, then build the framework, then migrate in batches without a big-bang rewrite.

[guoqqqi]

Completed under the updated migration boundary. The policy/docs work, Ginkgo bootstrap, first core-resource migration, and remaining command-family migration are all done and the related execution issues have been closed. Legacy mock Admin API unit-test removal is intentionally deferred and should be tracked separately only if we decide to remove them later.