Skip to content

feat(browserstack-service): test-level custom metadata hook handling (beforeEach/afterEach) for WDIO+Mocha (SDK-6843)#3

Open
anish353 wants to merge 1 commit into
mainfrom
test_level_custom_metadata
Open

feat(browserstack-service): test-level custom metadata hook handling (beforeEach/afterEach) for WDIO+Mocha (SDK-6843)#3
anish353 wants to merge 1 commit into
mainfrom
test_level_custom_metadata

Conversation

@anish353

Copy link
Copy Markdown
Owner

Proposed changes

Adds correct beforeEach / afterEach hook-timing handling for the browser.setCustomTags test-level custom-metadata feature that is already on main, for the WDIO + Mocha framework.

WDIO's Mocha runner fires user hooks outside the SDK's per-test tracking span:

  • beforeEach runs before beforeTest tracks the per-test instance, so a setCustomTags call there had no test to attach to (it hit the "outside of a resolvable test context; ignoring" path and was dropped).
  • afterEach runs after afterTest finishes the test and triggers the TestRunFinished (TEST/POST) send, so tags set in afterEach missed the payload.

This change makes hook-set tags land on the intended test — parity with the Java SDK's @Before / @After behavior:

  1. Buffer pre-test tags and flush at test start. Tags set before a per-test context exists (e.g. from beforeEach) are buffered in CustomTagsModule and flushed into the tracked test instance at test start, via a new onBeforeTest handler registered as a TEST / PRE observer. Then the buffer is reset per test.
  2. Route by the open Mocha hook window. The service layer (which sees WDIO's beforeHook / afterHook) records which Mocha hook window is open (before_each / after_each / before_all / after_all) so CustomTagsModule can route a setCustomTags call to the correct test — the CLI framework never sees Mocha's beforeEach / afterEach.
  3. Defer the TestRunFinished send past the after-each window. For Mocha, the TEST/POST send is deferred so tags set in afterEach still make the payload (the payload is serialized from the instance's live data map at send time). It is flushed at the next test's boundary, or from service.after() for the worker's last test.

Scope note: This is a standalone change on main, independent of the title-based Test-Case-ID PR (webdriverio#15378). When webdriverio#15378 merges it will need to reconcile with this change: both add an onBeforeTest method and a TEST / PRE observer to CustomTagsModule (this PR does the buffered-tag flush; webdriverio#15378 adds title-derived tagging). They should be merged into a single method + single observer registration rather than duplicated.

Jira: SDK-6843

Types of changes

  • Polish (an improvement for an existing feature)
  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update (improvements to the project's docs)
  • Specification changes (updates to WebDriver command specifications)
  • Internal updates (everything related to internal scripts, governance documentation and CI files)

Checklist

  • I have read the CONTRIBUTING doc
  • I have added tests that prove my fix is effective or that my feature works
  • I have added the necessary documentation (if appropriate)
  • I have added proper type definitions for new commands (if appropriate)

Live-verified end-to-end against a real BrowserStack Automate + Test Observability session (WDIO + Mocha, Chrome, v9). A spec sets testTagBefore in beforeEach, testTagInTest + testTagAfter in the test body, and testTagAfter again in afterEach, across two tests. Test 1's TestRunFinished custom_metadata came through as the correct union:

"custom_metadata": {
  "testTagBefore": { "field_type": "multi_dropdown", "values": ["Browserstack", "Before"] },
  "testTagInTest": { "field_type": "multi_dropdown", "values": ["Browserstack"] },
  "testTagAfter":  { "field_type": "multi_dropdown", "values": ["Browserstack", "After", "BrowserstackAfterChanged"] }
}

This proves both the beforeEach flush (testTagBefore landed) and the afterEach deferral (testTagAfter includes BrowserstackAfterChanged). Test 2 correctly received its own beforeEach tag and its afterEach tag was flushed at worker end via service.after().

Backport Request

  • This change is solely for v9 and doesn't need to be back-ported
  • Back-ported PR at #XXXXX

Further comments

The buffer and hook-window tracker are process-local (one module per worker) and Mocha runs tests serially within a worker, so a plain object / module-level variable is safe. The deferred-finish design stashes the finished test's own instance object (INIT_TEST replaces the tracked slot for the next test), so it stays valid across tests, and the payload is read fresh at send time so late merges are included.

Reviewers: @webdriverio/project-committers

…(beforeEach/afterEach) for WDIO+Mocha (SDK-6843)

Adds correct beforeEach/afterEach hook-timing handling for the setCustomTags
feature already on main. WDIO's Mocha runner fires user hooks OUTSIDE the SDK's
per-test tracking span: `beforeEach` runs before `beforeTest` tracks the test
instance, and `afterEach` runs after `afterTest` finishes it — so tags set in
those hooks previously either landed on the wrong test or missed the payload.

This change makes hook-set tags land on the intended test:
- Buffer tags set before a per-test context exists (e.g. from `beforeEach`) and
  flush them into the test at test start (a new TEST/PRE observer in
  CustomTagsModule that mints an onBeforeTest flush), parity with Java @before.
- The service layer records the open Mocha hook window (beforeHook/afterHook)
  so CustomTagsModule can route a setCustomTags call to the right test — the CLI
  framework never sees Mocha's beforeEach/afterEach.
- Defer the TestRunFinished (TEST/POST) send past the after-each hook window so
  tags set in `afterEach` still make the payload; the payload is serialized from
  the instance's live data at send time. Flushed at the next test's boundary, or
  from service.after() for the worker's last test.

Standalone on main, independent of the title-based-TC PR (webdriverio#15378). When webdriverio#15378
merges it will need to reconcile with this change: both add an onBeforeTest
method and a TEST/PRE observer to CustomTagsModule (this PR does the buffered-tag
flush; webdriverio#15378 adds title-derived tagging) — they must be merged into one method
and one observer registration rather than duplicated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant