From f447afe3c5b661f03dc6b0acf5e3c430fe4d8d48 Mon Sep 17 00:00:00 2001 From: Martin Thomson Date: Fri, 29 May 2026 17:28:27 +1000 Subject: [PATCH 1/3] Limitations section There is a lot of this document that talks about what it can do, but that fails to account for potential misapprehension about what is possible. This section attempts to enumerate limitations when it comes to using this API for the measurement of advertising effectiveness, particularly when it comes to producing information that is helpful in making decisions about where to invest in marketing. I've put this up front, so the disclaimer is clear. The section is longer than many of the adjoining sections; I hope that conveys the right sort of message. --- api.bs | 93 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) diff --git a/api.bs b/api.bs index a8540ca..15132fa 100644 --- a/api.bs +++ b/api.bs @@ -281,6 +281,90 @@ to split credit. --> +## Limitations and Successful Use ## {#limitations} + +The results that the Attribution API provides +can inform decisions about advertising strategies. +However, without due care, use of the API +could produce misleading results that support poor decisions. + +To make better decisions using the information provided by the API, +its limitations +especially the potential sources of bias and error, +need to be understood. +Understanding needs to be applied +both in the design of systems that use the API +and in the interpretation of any results. + +**Scope limitations**-- +A critical limitation is that only activity in the same browser +can be captured by the Attribution API. +The extent and effects of advertising campaigns +are rarely contained to a single medium. +This means that any measurement results obtained +cannot always capture all relevant activity. + +**Non-causal effects**-- +Some proportion of people who encounter advertising +already intend to follow through with the actions +the advertising seeks to encourage. +Attributing results with those advertisements +creates a false impression of their efficacy. + +To better measure the causal effect of advertising, +the use of (randomized) control trials-- +or, in industry terms, incrementality-- +is necessary [[JOHNSON20]]. +A basic advertising control trial +involves identifying a target audience, +advertising to a treatment group within that audience, +withholding advertising or showing default advertisements to a control group, +and measuring the difference in desired outcomes between the groups. + +This API can be used to support such trials. +This might be achieved by allocating control and treatment populations +to different histogram buckets. +Note however that the web platform presently does not have a facility for +providing consistent allocation of users +into control and treatment groups +across different sites. + +**Simple attribution logic**-- +The API provides a simple multi-touch attribution algorithm. +Simple attribution algorithms can introduce bias into results +in ways that effect decision-making. +Future versions of the API +might enable more sophisticated algorithms. + +**Sources of error**-- +There are multiple sources of error +in a distributed measurements system. +Differential privacy contributes some amount of noise to aggregates, +but this is a known quantity. + +Results might not accurately reflect what is being measured +for a range of other reasons, +some of which might be unavoidable, +including: + +* Failures and delays in propagating configuration to participating sites. +* Script errors that cause loss of impression or conversion events. +* Loss of stored impressions from storage pressure. +* Accidential or malicious recording of impressions. +* Network errors that result in lost reports. +* IVT classification that either + causes genuine events to be disregarded + or includes fraud and accidental mistakes. + +Errors such as these could add an unknown amount of uncertainty +to measurement results. +Sites that use the API are responsible +for managing their exposure to these and other classes of error. +Future versions of the API +could seek to provide better assistance to sites +in handling errors. + + # Overview of Operation # {#overview} The Attribution API provides aggregate information about the @@ -3985,6 +4069,15 @@ spec:css-2025; type:dfn; text:user "publisher": "Bureau of Economic Analysis", "date": "2017-10" }, + "johnson20": { + "authors": [ + "Garrett A. Johnson" + ], + "title": "Inferno: A Guide to Field Experiments in Online Display Advertising", + "href": "https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3581396", + "publisher": "Journal of Economics & Management Strategy", + "date": "2023-01-18" + }, "online-advertising": { "authors": [ "Avi Goldfarb", From b4863ff28f0a6227040d99d07927b01cc87307b2 Mon Sep 17 00:00:00 2001 From: Martin Thomson Date: Fri, 29 May 2026 17:43:06 +1000 Subject: [PATCH 2/3] Typos, formatting --- api.bs | 146 ++++++++++++++++++++++++++++++--------------------------- 1 file changed, 76 insertions(+), 70 deletions(-) diff --git a/api.bs b/api.bs index 15132fa..c8a1dfb 100644 --- a/api.bs +++ b/api.bs @@ -289,80 +289,86 @@ However, without due care, use of the API could produce misleading results that support poor decisions. To make better decisions using the information provided by the API, -its limitations -especially the potential sources of bias and error, +its limitations-- +especially the potential sources of bias and error-- need to be understood. Understanding needs to be applied both in the design of systems that use the API and in the interpretation of any results. -**Scope limitations**-- -A critical limitation is that only activity in the same browser -can be captured by the Attribution API. -The extent and effects of advertising campaigns -are rarely contained to a single medium. -This means that any measurement results obtained -cannot always capture all relevant activity. - -**Non-causal effects**-- -Some proportion of people who encounter advertising -already intend to follow through with the actions -the advertising seeks to encourage. -Attributing results with those advertisements -creates a false impression of their efficacy. - -To better measure the causal effect of advertising, -the use of (randomized) control trials-- -or, in industry terms, incrementality-- -is necessary [[JOHNSON20]]. -A basic advertising control trial -involves identifying a target audience, -advertising to a treatment group within that audience, -withholding advertising or showing default advertisements to a control group, -and measuring the difference in desired outcomes between the groups. - -This API can be used to support such trials. -This might be achieved by allocating control and treatment populations -to different histogram buckets. -Note however that the web platform presently does not have a facility for -providing consistent allocation of users -into control and treatment groups -across different sites. - -**Simple attribution logic**-- -The API provides a simple multi-touch attribution algorithm. -Simple attribution algorithms can introduce bias into results -in ways that effect decision-making. -Future versions of the API -might enable more sophisticated algorithms. - -**Sources of error**-- -There are multiple sources of error -in a distributed measurements system. -Differential privacy contributes some amount of noise to aggregates, -but this is a known quantity. - -Results might not accurately reflect what is being measured -for a range of other reasons, -some of which might be unavoidable, -including: - -* Failures and delays in propagating configuration to participating sites. -* Script errors that cause loss of impression or conversion events. -* Loss of stored impressions from storage pressure. -* Accidential or malicious recording of impressions. -* Network errors that result in lost reports. -* IVT classification that either - causes genuine events to be disregarded - or includes fraud and accidental mistakes. - -Errors such as these could add an unknown amount of uncertainty -to measurement results. -Sites that use the API are responsible -for managing their exposure to these and other classes of error. -Future versions of the API -could seek to provide better assistance to sites -in handling errors. +
+ +: Scope limitations +:: A critical limitation is that only activity in the same browser + can be captured by the Attribution API. + The extent and effects of advertising campaigns + are rarely contained to a single medium. + This means that any measurement obtained + cannot be assumed to capture all relevant activity. + +: Non-causal effects +:: Some proportion of people who encounter advertising + already intend to follow through with the actions + the advertising seeks to encourage. + Attributing outcomes to those advertisements + creates a false impression of their efficacy. + +:: To better measure the causal effect of advertising, + the use of (randomized) control trials-- + or, in industry terms, incrementality-- + is necessary [[INFERNO]]. + A basic advertising control trial + involves identifying a target audience, + advertising to a treatment group within that audience, + withholding advertising or showing default advertisements to a control group, + and measuring the difference in desired outcomes between the groups. + +:: This API can be used to support such trials. + This might be achieved by allocating control and treatment populations + to different histogram buckets. + Note however that the web platform presently does not have a facility for + providing consistent allocation of users + into control and treatment groups + across different sites. + +: Simple attribution logic +:: The API provides a simple multi-touch attribution algorithm. + Simple attribution algorithms can introduce bias into results + in ways that affect decision-making. + Future versions of the API + might enable more sophisticated algorithms. + +: Sources of error +:: There are multiple sources of error + in a distributed measurement system. + Differential privacy contributes some amount of noise to aggregates, + but this is a known quantity. + +:: Results might not accurately reflect what is being measured + for a range of other reasons, + some of which might be unavoidable, + including: + + * Failures and delays in propagating configuration to participating sites. + * Script errors that cause loss of impression or conversion events. + * Loss of stored impressions from storage pressure. + * Accidental or malicious recording of impressions. + * Network errors that result in lost reports. + * IVT classification that + causes genuine events to be disregarded. + * IVT classification that + erroneously includes fraud or accidental mistakes. + + +:: Errors such as these could add an unknown amount of uncertainty + to measurement results. + Sites that use the API are responsible + for managing their exposure to these and other classes of error. + Future versions of the API + could seek to provide better assistance to sites + in handling errors. + +
# Overview of Operation # {#overview} @@ -4069,7 +4075,7 @@ spec:css-2025; type:dfn; text:user "publisher": "Bureau of Economic Analysis", "date": "2017-10" }, - "johnson20": { + "inferno": { "authors": [ "Garrett A. Johnson" ], From d4ae7d949016b8ce8ea4cc65bb5f4de21e022ea6 Mon Sep 17 00:00:00 2001 From: Martin Thomson Date: Mon, 1 Jun 2026 13:54:42 +1000 Subject: [PATCH 3/3] Small changes outside the limitations section These are designed to avoid overegging the pudding, by implying that simple attribution (comparing sites or creatives) is the entire story. --- api.bs | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/api.bs b/api.bs index c8a1dfb..12cccd1 100644 --- a/api.bs +++ b/api.bs @@ -98,7 +98,7 @@ The cost of measurement performance was privacy. In order to produce accurate and comprehensive information, advertising businesses performed extensive tracking of the activity of all Web users. Each browser was given a tracking identifier, -often using cookies that were logged by cross-site content. +often using cookies that associated with cross-site resources. Every action of interest was logged against this identifier, forming a comprehensive record of a person's online activities. @@ -156,7 +156,7 @@ Participation in an [=attribution=] measurement system would comprise a secondary cost to Web users. Support for attribution enables more effective advertising, -largely by informing advertisers about what ads perform best, +largely by helping advertisers understand which ads perform best, and in what circumstances. Those circumstances might include the time and place that the ad is shown, @@ -166,7 +166,8 @@ the details of the ad itself. Connecting that information to outcomes allows an advertiser to learn what circumstances most often lead to the outcomes they most value. -That allows advertisers to spend more on effective advertising +When [[#limitations|attribution is used effectively]], +it allows advertisers to spend more on effective advertising and less on ineffective advertising. This lowers the overall cost of advertising relative to the value obtained. [[ONLINE-ADVERTISING]] @@ -250,6 +251,7 @@ and the tallies of conversions attributed to each form a histogram. Each bucket of the histogram counts the conversions for a group of ads. +This allows for comparison across the groupings. 
@@ -260,6 +262,9 @@ path:images/histogram.svg
Different groupings might be used for different purposes. +This enables both +rigorous [[#limitations|experiments that seek to establish causality]] +and simpler comparisons of different treatments. For instance, grouping by creative (the content of an ad) might be used to learn which creative works best.