Limitations section

api.bs

@@ -98,7 +98,7 @@ The cost of measurement performance was privacy.
 In order to produce accurate and comprehensive information,
 advertising businesses performed extensive tracking of the activity of all Web users.
 Each browser was given a tracking identifier,
-often using cookies that were logged by cross-site content.
+often using cookies that associated with cross-site resources.
 Every action of interest was logged against this identifier,
 forming a comprehensive record of a person's online activities.
 
@@ -156,7 +156,7 @@ Participation in an [=attribution=] measurement system
 would comprise a secondary cost to Web users.
 
 Support for attribution enables more effective advertising,
-largely by informing advertisers about what ads perform best,
+largely by helping advertisers understand which ads perform best,
 and in what circumstances.
 Those circumstances might include
 the time and place that the ad is shown,
@@ -166,7 +166,8 @@ the details of the ad itself.
 Connecting that information to outcomes
 allows an advertiser to learn what circumstances most often lead
 to the outcomes they most value.
-That allows advertisers to spend more on effective advertising
+When [[#limitations|attribution is used effectively]],
+it allows advertisers to spend more on effective advertising
 and less on ineffective advertising.
 This lowers the overall cost of advertising
 relative to the value obtained. [[ONLINE-ADVERTISING]]
@@ -250,6 +251,7 @@ and the tallies of conversions attributed to each
 form a histogram.
 Each bucket of the histogram counts the conversions
 for a group of ads.
+This allows for comparison across the groupings.
 
 <figure>
 <pre class=include-raw>
@@ -260,6 +262,9 @@ path:images/histogram.svg
 </figure>
 
 Different groupings might be used for different purposes.
+This enables both
+rigorous [[#limitations|experiments that seek to establish causality]]
+and simpler comparisons of different treatments.
 For instance, grouping by creative (the content of an ad)
 might be used to learn which creative works best.
 
@@ -281,6 +286,96 @@ to split credit.
 -->
 
 
+## Limitations and Successful Use ## {#limitations}
+
+The results that the Attribution API provides
+can inform decisions about advertising strategies.
+However, without due care, use of the API
+could produce misleading results that support poor decisions.
+
+To make better decisions using the information provided by the API,
+its limitations--
+especially the potential sources of bias and error--
+need to be understood.
+Understanding needs to be applied
+both in the design of systems that use the API
+and in the interpretation of any results.
+
+<dl>
+
+: Scope limitations
+::  A critical limitation is that only activity in the same browser
+    can be captured by the Attribution API.
+    The extent and effects of advertising campaigns
+    are rarely contained to a single medium.
+    This means that any measurement obtained
+    cannot be assumed to capture all relevant activity.
+
+: Non-causal effects
+::  Some proportion of people who encounter advertising
+    already intend to follow through with the actions
+    the advertising seeks to encourage.
+    Attributing outcomes to those advertisements
+    creates a false impression of their efficacy.
+
+::  To better measure the causal effect of advertising,
+    the use of (randomized) control trials--
+    or, in industry terms, incrementality--
+    is necessary [[INFERNO]].
+    A basic advertising control trial
+    involves identifying a target audience,
+    advertising to a treatment group within that audience,
+    withholding advertising or showing default advertisements to a control group,
+    and measuring the difference in desired outcomes between the groups.
+
+::  This API can be used to support such trials.
+    This might be achieved by allocating control and treatment populations
+    to different histogram buckets.
+    Note however that the web platform presently does not have a facility for
+    providing consistent allocation of users
+    into control and treatment groups
+    across different sites.
+
+: Simple attribution logic
+::  The API provides a simple multi-touch attribution algorithm.
+    Simple attribution algorithms can introduce bias into results
+    in ways that affect decision-making.
+    Future versions of the API
+    might enable more sophisticated algorithms.
+
+: Sources of error
+::  There are multiple sources of error
+    in a distributed measurement system.
+    Differential privacy contributes some amount of noise to aggregates,
+    but this is a known quantity.
+
+::   Results might not accurately reflect what is being measured
+    for a range of other reasons,
+    some of which might be unavoidable,
+    including:
+
+    *   Failures and delays in propagating configuration to participating sites.
+    *   Script errors that cause loss of impression or conversion events.
+    *   Loss of stored impressions from storage pressure.
+    *   Accidental or malicious recording of impressions.
+    *   Network errors that result in lost reports.
+    *   <abbr title="invalid traffic">IVT</abbr> classification that
+        causes genuine events to be disregarded.
+    *   <abbr title="invalid traffic">IVT</abbr> classification that
+        erroneously includes fraud or accidental mistakes.
+
+
+::  Errors such as these could add an unknown amount of uncertainty
+    to measurement results.
+    Sites that use the API are responsible
+    for managing their exposure to these and other classes of error.
+    Future versions of the API
+    could seek to provide better assistance to sites
+    in handling errors.
+
+</dl>
+
+
 # Overview of Operation # {#overview}
 
 The Attribution API provides aggregate information about the
@@ -3985,6 +4080,15 @@ spec:css-2025; type:dfn; text:user
     "publisher": "Bureau of Economic Analysis",
     "date": "2017-10"
   },
+  "inferno": {
+    "authors": [
+      "Garrett A. Johnson"
+    ],
+    "title": "Inferno: A Guide to Field Experiments in Online Display Advertising",
+    "href": "https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3581396",
+    "publisher": "Journal of Economics & Management Strategy",
+    "date": "2023-01-18"
+  },
   "online-advertising": {
     "authors": [
       "Avi Goldfarb",