fix(browserstack-service): land beforeEach/afterEach custom tags in WDIO+Mocha (SDK-6843)#2
Closed
anish353 wants to merge 4 commits into
Conversation
…IO+Mocha (SDK-6843) Add build-level scope to setCustomTags via a third arg: setCustomTags(key, value, true). Build-level tags ride the build-finish (stopBinSession.custom_metadata) payload instead of the per-test event. Since WDIO runs one process per spec file with no shared memory, each worker persists a full snapshot to os.tmpdir()/bstack_build_tags_<runId>_<pid>.json and the main process aggregates them at build finish (mirrors node-agent). Binary/CLI flow only; Direct/Listener-flow build-level calls no-op with a warning. Reuses the existing tokenizer + merge engine; mocha-gating preserved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…(SDK-6843)
Test-level browser.setCustomTags calls made from a beforeEach hook were dropped ('called outside a resolvable test context'): WDIO runs beforeEach before beforeTest establishes the tracked test instance, so getTrackedInstance() was null. Buffer such pre-test tags in a per-worker pendingTestLevelTags store and flush them into the test at test start (onBeforeTest / TEST PRE), then reset per-test. This gives parity with Java @before tags. Scope: beforeEach, Binary/CLI flow. afterEach (dispatch precedes it) and the legacy Direct/Listener handler are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…beforeEach fix (SDK-6843) Build-level custom_metadata (the setCustomTags 3rd-arg -> build custom_metadata) is not a documented BrowserStack feature. The documented build-level tagging path is 'build tags' (config-time buildTag in wdio.conf.js / browserstack.yml, a flat string list for build filtering), which WDIO already supports. Reverting the build-level additions in customTags.ts, cli/index.ts, cli/grpcClient.ts, custom-tags-handler.ts, and the public type, and removing the build-level unit tests. Retains ONLY the beforeEach test-level fix in customTagsModule.ts: buffer pre-test setCustomTags calls and flush them into the test at test start. That addresses a documented feature (test-level custom metadata) whose beforeEach-hook path was previously dropped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…SDK-6843) WDIO dispatches the TestRunFinished (TEST/POST) send at afterTest, which fires BEFORE the user's afterEach hooks — so browser.setCustomTags calls made in afterEach merged into local state but never reached the wire. Fix defers only the SEND past the after-each hook window (mocha-gated): TestHubModule stashes the TEST/POST send and flushes it at the next test's first event (each test gets a fresh instance, so the stashed reference stays valid) or at worker end via service.after(). The payload serializes from the instance's live data at send time, so late tag merges are included; uuid/started_at/ended_at are already stamped in the data and do not drift, and an explicit state override keeps the event labeled TEST/POST even if LOG events moved the instance's current state. The service layer now records which Mocha hook window is open (beforeHook classifies the hook title; afterHook clears), and CustomTagsModule routes tags by window: before-each/before-all buffer for the UPCOMING test (also fixing a latent bug where a 2nd+ test's beforeEach tags merged into the previous test's already-sent instance), after-each/after-all/in-test merge into the current test. Live-verified on a 2-test spec: body+afterEach same-key union, per-test hook-tag routing with zero cross-test leakage, and exactly one TestRunFinished per test (no drops). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Owner
Author
|
Superseded by #3, which re-bases this change onto |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
Fixes test-level custom-metadata hook timing in WebdriverIO + Mocha:
browser.setCustomTags(key, value)called frombeforeEachorafterEachhooks was lost — only in-test-body calls reached the test'sTestRunFinishedevent. With this PR, tags set inbeforeEach, the test body, andafterEachall land on the test (parity with the Java SDK, where@Before/@Aftertags work).Root causes
beforeEach: WDIO runs it beforebeforeTestestablishes the tracked per-test instance, so the call had no test context and was dropped ("called outside a resolvable test context").afterEach: WDIO dispatches theTestRunFinished(TEST/POST) send atafterTest, which fires before the user'safterEach— the tag merged into local state after the event was already sent.Fix (Binary/CLI flow, mocha-gated)
customTags.ts,service.ts): the service'sbeforeHook/afterHookrecord which Mocha hook window is open (before each/after each/before all/after all, classified from the hook title) — the CLI framework never sees these hooks by itself.cli/modules/customTagsModule.ts): tags set in a before-each/before-all window buffer for the upcoming test and flush at test start; tags set in after-each/after-all or the test body merge into the current test. (This also fixes a latent bug where a 2nd+ test'sbeforeEachtags silently merged into the previous test's already-sent instance.)cli/modules/testHubModule.ts,service.ts): the TEST/POSTTestRunFinishedsend is stashed instead of sent atafterTest, and flushed at the next test's first event (each test gets a fresh instance, so the stashed reference stays valid) or at worker end viaservice.after(). The payload serializes from the instance's live data at send time, soafterEachmerges are included; uuid/started_at/ended_atare already stamped in the data (no drift), and an explicit state override keeps the event labeled TEST/POST even if LOG events move the instance's current state.Scope: Binary/CLI flow. The legacy Direct/Listener handler is unchanged.
Jira: SDK-6843.
Types of changes
Checklist
setCustomTagsstays(key, value))Backport Request
v9and doesn't need to be back-ported#XXXXXFurther comments
Why defer the send instead of re-sending:
TestRunFinishedis one-shot per test; re-sending would risk duplicate test runs. Deferring only the transmission (not the event bookkeeping) keeps every other TEST/POST observer (Automate session status, accessibility, Percy) running atafterTestexactly as before — only the TestHub gRPC send moves past the hook window.Verification (live, real BrowserStack sessions, 2-test spec sharing beforeEach/afterEach):
custom_metadataon the wire:testTagBefore=["Browserstack","Before"](beforeEach),testTagInTest=["Browserstack"](body), and the same-key uniontestTagAfter=["Browserstack","After","BrowserstackAfterChanged"](body + afterEach, existing-first order).beforeEach/afterEachtags (testTagBefore,testTagAfter=["BrowserstackAfterChanged","After"]via the worker-end flush) with zero cross-test leakage in either direction.TestRunFinishedper test reached the binary (deferral drops nothing); the deferral/flush debug trail shows 2× stash + 2× flush (next-test boundary and worker end).Reviewers: @webdriverio/project-committers
🤖 Generated with Claude Code