Skip to content

Edge Event Batching implementation - #199

Open
sagar-sharma-adobe wants to merge 10 commits into
mainfrom
dev_batching
Open

Edge Event Batching implementation#199
sagar-sharma-adobe wants to merge 10 commits into
mainfrom
dev_batching

Conversation

@sagar-sharma-adobe

@sagar-sharma-adobe sagar-sharma-adobe commented Jul 15, 2026

Copy link
Copy Markdown

Description

Summary

Adds Experience Event batching to the Edge extension: consecutive ExperienceEvents that share a snapshotted request config can now be coalesced into a single Edge Network request instead of one request per event, reducing network overhead for high-volume event sources (e.g. Optimize/Target tracking, Messaging interactions). Batching is opt-in and off by default — SDKs that never enable it see byte-for-byte identical behavior to today.

Also adds a per-event error callback: EdgeCallbackWithError / EdgeEventError, delivered via a new Edge.sendEvent(ExperienceEvent, EdgeCallbackWithError) overload, so callers can receive Edge Network errors for their specific event, not just success handles.

Configuration

Two new keys, both read from the Configuration shared state:

  • edge.batching.enabled (bool, default false)
  • edge.batching.eventNameAllowlist (list of event names) — batching is opt-in per event name; an event only batches if its name is explicitly allowlisted, even when enabled is true.

Both keys also support a first-launch fallback: an optional adb_edgeBatchingConfig.json bundled in the app's assets (EdgeBundledBatchingConfig), consulted per-key only when that key is absent from the Configuration shared state. Anything set via MobileCore.updateConfiguration() or a remote/Launch config always wins over the bundled file for that key.

How batching works

  • Queue: EdgeBatchingHitQueue replaces PersistentHitQueue as the Edge extension's hit queue. Each cycle peeks a window sized from the head entity's snapshotted config: 1 if batching is disabled (identical to today), or min(queueDepth, MAX_BATCH_SIZE=10) if enabled.
  • Eligibility: EdgeHitProcessor.processBatch walks the peeked window and only folds in consecutive entities that are all ExperienceEvents, all on the allowlist, and all share the head's config (a batched request can only carry one datastream ID/override — an entity with a different config truncates the run and becomes its own head on a later cycle). Hitting a Consent/Reset event, a non-allowlisted name, or a config mismatch stops the run; anything before that point still batches.
  • 400 handling: a 400 on a real (N>1) batch means nothing was ingested. Rather than resend the whole batch, the processor signals BatchOutcome.explode(N) and the queue drains those N entities one at a time through the exact same single-event send path non-batching builds already use, retrying a recoverable failure in place (never skipping it). This is not the same as before batching existed: on 400/other unrecoverable codes, isBatchRequest on EdgeNetworkService.doRequest gates whether the network layer classifies a 400 as explodable — only real batches take that branch; a single event's 400 falls through to the identical unrecoverable-error handling as pre-batching code.
  • Per-event completion: NetworkResponseHandler now completes each event in a batch as soon as a higher-indexed response fragment arrives, instead of waiting for the whole request's stream to close — so an early event in a batch doesn't wait on a later one. A handle/error with no eventIndex is treated as request-scoped: for state:store/locationHint:result (session-global side effects) it's broadcast to all waiting events; for errors, a root-level error is treated as the whole batch failing, so it fans out to every waiting event's callback. Single events are unaffected — with one waiting event, this degrades to exactly today's behavior.

Notable internal refactors (behavior-preserving)

  • BatchOutcome is a small Kind (DONE/RETRY/EXPLODE) + single int value, rather than three independent counters where only one was ever meaningful.
  • RetryResult.shouldRetry is now computed from NetworkRequestOutcome rather than stored redundantly.
  • The batch and single-event request-build/send paths share buildExperienceEventHit/sendEdgeHit instead of duplicating logic.
  • Non-batching log messages/format are preserved as-is (including event IDs) versus main.

Compatibility

  • Batching off (default): queue window is always 1, processBatch routes every entity through the pre-existing single-event path, and 400/error/retry handling is unchanged from before this feature.
  • New public API only adds: EdgeCallbackWithError, EdgeEventError, and the new Edge.sendEvent overload — no existing signatures changed.

Testing

  • Unit: 473 tests across EdgeHitProcessor, EdgeBatchingHitQueue, NetworkResponseHandler, EdgeNetworkService, EventUtils, EdgeBundledBatchingConfig, CompletionCallbacksManager — batch formation, config-mismatch truncation, 400/explode/drain (including retry-without-skip), per-event completion ordering, global-handle/error broadcast, and bundled-config precedence.
  • Functional (androidTest, on-device/emulator): new EdgeBatchingFunctionalTests (non-batching regression, allowlist gating, real batch coalescing, per-event completion under a server error) plus updates to NetworkResponseHandlerFunctionalTests and CompletionHandlerFunctionalTests for the new global-handle/error-fanout semantics and the EdgeCallbackWithError path.
  • On-device, real Edge Network traffic (Optimize test app): verified real batch formation (multiple tracking events coalesced into one request with correct per-index handle attribution), global no-index handles (state:store, locationHint:result) broadcasting correctly in a real multi-event response, the non-batching path producing one request per event with batching off, and a real per-event error response — with no crashes, warnings, or response-ordering violations across all scenarios.

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • I have signed the Adobe Open Source CLA.
  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

…pline

BatchOutcome.done() now carries a resolvedCount so EdgeBatchingHitQueue only
removes what processBatch actually resolved, instead of always removing the
full peeked window — closing a bug where truncated batches (Consent/Reset
boundary, decode failures, single non-ExperienceEvent head) silently dropped
untouched trailing entities. Also flips earlyPerEventCompletionEnabled to true
by default so the completion(N)-before-handle(N+1) ordering guarantee is
actually active in production, matching its own doc comment.
- 400 handling: single-event requests (incl. exploded resends and
  Consent/Reset) now match main's outcome and code path exactly, via an
  isBatchRequest flag on doRequest; only real N>1 batches explode. Batch
  400s drain one event at a time through the single-event path, retrying
  in place without skipping.
- Only batch consecutive events that share the head's snapshotted config.
- Model BatchOutcome as Kind + value; make RetryResult.shouldRetry
  computed; drop the earlyPerEventCompletion flag (now default); dedup the
  batch/single request-build and send paths; preserve main's log messages.
- Tests cover the new paths; full Edge suite green (468 tests).
@sagar-sharma-adobe
sagar-sharma-adobe marked this pull request as ready for review August 10, 2026 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant