Skip to content

fix(browserstack-service): land beforeEach/afterEach custom tags in WDIO+Mocha (SDK-6843)#2

Closed
anish353 wants to merge 4 commits into
feat/wdio-mocha-title-based-tc-idfrom
feature/SDK-6843-wdio-build-level-custom-tags
Closed

fix(browserstack-service): land beforeEach/afterEach custom tags in WDIO+Mocha (SDK-6843)#2
anish353 wants to merge 4 commits into
feat/wdio-mocha-title-based-tc-idfrom
feature/SDK-6843-wdio-build-level-custom-tags

Conversation

@anish353

@anish353 anish353 commented Jul 10, 2026

Copy link
Copy Markdown
Owner

Proposed changes

Fixes test-level custom-metadata hook timing in WebdriverIO + Mocha: browser.setCustomTags(key, value) called from beforeEach or afterEach hooks was lost — only in-test-body calls reached the test's TestRunFinished event. With this PR, tags set in beforeEach, the test body, and afterEach all land on the test (parity with the Java SDK, where @Before/@After tags work).

Root causes

  • beforeEach: WDIO runs it before beforeTest establishes the tracked per-test instance, so the call had no test context and was dropped ("called outside a resolvable test context").
  • afterEach: WDIO dispatches the TestRunFinished (TEST/POST) send at afterTest, which fires before the user's afterEach — the tag merged into local state after the event was already sent.

Fix (Binary/CLI flow, mocha-gated)

  • Hook-window tracking (customTags.ts, service.ts): the service's beforeHook/afterHook record which Mocha hook window is open (before each / after each / before all / after all, classified from the hook title) — the CLI framework never sees these hooks by itself.
  • Routing (cli/modules/customTagsModule.ts): tags set in a before-each/before-all window buffer for the upcoming test and flush at test start; tags set in after-each/after-all or the test body merge into the current test. (This also fixes a latent bug where a 2nd+ test's beforeEach tags silently merged into the previous test's already-sent instance.)
  • Deferred send (cli/modules/testHubModule.ts, service.ts): the TEST/POST TestRunFinished send is stashed instead of sent at afterTest, and flushed at the next test's first event (each test gets a fresh instance, so the stashed reference stays valid) or at worker end via service.after(). The payload serializes from the instance's live data at send time, so afterEach merges are included; uuid/started_at/ended_at are already stamped in the data (no drift), and an explicit state override keeps the event labeled TEST/POST even if LOG events move the instance's current state.

Scope: Binary/CLI flow. The legacy Direct/Listener handler is unchanged.

Jira: SDK-6843.

Types of changes

  • Polish (an improvement for an existing feature)
  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update (improvements to the project's docs)
  • Specification changes (updates to WebDriver command specifications)
  • Internal updates (everything related to internal scripts, governance documentation and CI files)

Checklist

  • I have read the CONTRIBUTING doc
  • I have added tests that prove my fix is effective or that my feature works — verified live end-to-end (see below); no unit-test hook exists for the module's framework-tracking path
  • I have added the necessary documentation (if appropriate) — N/A
  • I have added proper type definitions for new commands (if appropriate) — N/A (no API-signature change; setCustomTags stays (key, value))

Backport Request

  • This change is solely for v9 and doesn't need to be back-ported
  • Back-ported PR at #XXXXX

Further comments

Why defer the send instead of re-sending: TestRunFinished is one-shot per test; re-sending would risk duplicate test runs. Deferring only the transmission (not the event bookkeeping) keeps every other TEST/POST observer (Automate session status, accessibility, Percy) running at afterTest exactly as before — only the TestHub gRPC send moves past the hook window.

Verification (live, real BrowserStack sessions, 2-test spec sharing beforeEach/afterEach):

  • Test 1 custom_metadata on the wire:
    testTagBefore=["Browserstack","Before"] (beforeEach), testTagInTest=["Browserstack"] (body), and the same-key union testTagAfter=["Browserstack","After","BrowserstackAfterChanged"] (body + afterEach, existing-first order).
  • Test 2 got its own beforeEach/afterEach tags (testTagBefore, testTagAfter=["BrowserstackAfterChanged","After"] via the worker-end flush) with zero cross-test leakage in either direction.
  • Exactly one TestRunFinished per test reached the binary (deferral drops nothing); the deferral/flush debug trail shows 2× stash + 2× flush (next-test boundary and worker end).

History note: earlier commits on this branch added, then removed, build-level custom metadata (setCustomTags(k, v, true)) — dropped because build-level custom_metadata is not a documented BrowserStack feature (the documented build-level path is the buildTag config, which WDIO already supports). The PR's net diff is the test-level hook-timing fixes only. Squash-merge recommended.

Reviewers: @webdriverio/project-committers

🤖 Generated with Claude Code

anish353 and others added 3 commits July 10, 2026 08:37
…IO+Mocha (SDK-6843)

Add build-level scope to setCustomTags via a third arg: setCustomTags(key, value, true). Build-level tags ride the build-finish (stopBinSession.custom_metadata) payload instead of the per-test event. Since WDIO runs one process per spec file with no shared memory, each worker persists a full snapshot to os.tmpdir()/bstack_build_tags_<runId>_<pid>.json and the main process aggregates them at build finish (mirrors node-agent). Binary/CLI flow only; Direct/Listener-flow build-level calls no-op with a warning. Reuses the existing tokenizer + merge engine; mocha-gating preserved.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…(SDK-6843)

Test-level browser.setCustomTags calls made from a beforeEach hook were dropped ('called outside a resolvable test context'): WDIO runs beforeEach before beforeTest establishes the tracked test instance, so getTrackedInstance() was null. Buffer such pre-test tags in a per-worker pendingTestLevelTags store and flush them into the test at test start (onBeforeTest / TEST PRE), then reset per-test. This gives parity with Java @before tags. Scope: beforeEach, Binary/CLI flow. afterEach (dispatch precedes it) and the legacy Direct/Listener handler are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…beforeEach fix (SDK-6843)

Build-level custom_metadata (the setCustomTags 3rd-arg -> build custom_metadata) is not a documented BrowserStack feature. The documented build-level tagging path is 'build tags' (config-time buildTag in wdio.conf.js / browserstack.yml, a flat string list for build filtering), which WDIO already supports. Reverting the build-level additions in customTags.ts, cli/index.ts, cli/grpcClient.ts, custom-tags-handler.ts, and the public type, and removing the build-level unit tests.

Retains ONLY the beforeEach test-level fix in customTagsModule.ts: buffer pre-test setCustomTags calls and flush them into the test at test start. That addresses a documented feature (test-level custom metadata) whose beforeEach-hook path was previously dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@anish353 anish353 changed the title feat(browserstack-service): build-level custom metadata for WebdriverIO+Mocha (SDK-6843) fix(browserstack-service): land beforeEach custom tags in WDIO+Mocha (SDK-6843) Jul 13, 2026
…SDK-6843)

WDIO dispatches the TestRunFinished (TEST/POST) send at afterTest, which fires BEFORE the user's afterEach hooks — so browser.setCustomTags calls made in afterEach merged into local state but never reached the wire. Fix defers only the SEND past the after-each hook window (mocha-gated): TestHubModule stashes the TEST/POST send and flushes it at the next test's first event (each test gets a fresh instance, so the stashed reference stays valid) or at worker end via service.after(). The payload serializes from the instance's live data at send time, so late tag merges are included; uuid/started_at/ended_at are already stamped in the data and do not drift, and an explicit state override keeps the event labeled TEST/POST even if LOG events moved the instance's current state.

The service layer now records which Mocha hook window is open (beforeHook classifies the hook title; afterHook clears), and CustomTagsModule routes tags by window: before-each/before-all buffer for the UPCOMING test (also fixing a latent bug where a 2nd+ test's beforeEach tags merged into the previous test's already-sent instance), after-each/after-all/in-test merge into the current test. Live-verified on a 2-test spec: body+afterEach same-key union, per-test hook-tag routing with zero cross-test leakage, and exactly one TestRunFinished per test (no drops).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@anish353 anish353 changed the title fix(browserstack-service): land beforeEach custom tags in WDIO+Mocha (SDK-6843) fix(browserstack-service): land beforeEach/afterEach custom tags in WDIO+Mocha (SDK-6843) Jul 13, 2026
@anish353

Copy link
Copy Markdown
Owner Author

Superseded by #3, which re-bases this change onto main so test-level custom-metadata hook handling can release independently of the title-based-TC PR (webdriverio#15378). This branch's name ("build-level") was also misleading — build-level was dropped. Closing in favor of #3 (branch test_level_custom_metadata).

@anish353 anish353 closed this Jul 14, 2026
@anish353 anish353 deleted the feature/SDK-6843-wdio-build-level-custom-tags branch July 14, 2026 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant