Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 24 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,22 +34,22 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for the contribution workflow. The cross-

## First-party event measurement

Cloudflare D1 stores four anonymous website event types:
Cloudflare D1 stores four website event types:

- `page_viewed`
- `download_clicked`
- `download_failed`
- `outbound_link_clicked`

The implementation intentionally stores no IP address, cookie, fingerprint, referrer, or persistent visitor identifier. Counts therefore represent events rather than unique visitors. `download_failed` is limited to a failure in ScientFactory's redirect service; the website cannot observe a transfer failure after GitHub begins serving an installer.
Before a visitor chooses analytics, and after choosing **Essential only**, every event receives a new event-specific identifier. Those counts represent events rather than unique people. After explicit **Allow analytics** consent, the site sets a random first-party visitor identifier and creates a session identifier so visits, downloads, sessions, and return behavior can be measured. The identity is not derived from an IP address, browser fingerprint, email address, advertising identifier, referrer, or third-party account. `download_failed` is limited to a failure in ScientFactory's redirect service; the website cannot observe a transfer failure after GitHub begins serving an installer.

The production binding is `DOWNLOAD_DB`, backed by the `scientfactory-downloads` D1 database. New events use the shared `analytics_events` table; the earlier `site_events` table remains as read-only historical data. Apply new migrations before deploying code that depends on them:

```sh
bun run db:migrate
```

To view the lifetime event summary plus 30-day download, outbound-link, and failure breakdowns:
To view the lifetime event summary plus 30-day identity, consent, session, download, outbound-link, and failure breakdowns:

```sh
bun run analytics:report
Expand All @@ -59,7 +59,9 @@ Local and Cloudflare preview hosts do not write events, which keeps production c

## Analytics gateway

The Worker under `workers/events` is ScientFactory's first-party telemetry gateway. Desktop clients submit bounded event batches to `https://events.scientfactory.com/v1/events`; the Worker stores them in D1 first and can then forward anonymous copies to the ScientFactory EU PostHog project. PostHog is an optional analysis layer rather than the primary event store.
The Worker under `workers/events` is ScientFactory's first-party telemetry and identity gateway. Desktop clients submit bounded event batches to `https://events.scientfactory.com/v1/events`; the Worker stores them in D1 first and can then forward pseudonymous copies to the ScientFactory EU PostHog project. PostHog is an optional analysis layer rather than the primary event store.

Website visitors, desktop installations, sessions, and future Scient accounts use separate opaque identifiers. The service-authenticated `POST /v1/identity/link` endpoint can connect a visitor or installation to an account after Scient's account service has authenticated that user. Browser and desktop clients cannot call this endpoint directly or claim an account identifier. Linking updates first-party historical events without changing the user's analytics choice; the corresponding anonymous-to-account PostHog identity event is forwarded only for product-or-higher consent.

Generate binding types and validate the Worker with:

Expand All @@ -74,4 +76,21 @@ Deploy the Worker only from an approved production change:
bun run events:deploy
```

`POSTHOG_PROJECT_TOKEN` is a Cloudflare Worker secret and must never be committed. If it is absent, ingestion continues and events remain queued in D1 for later delivery.
`POSTHOG_PROJECT_TOKEN` and `IDENTITY_LINK_TOKEN` are Cloudflare Worker secrets and must never be committed. If the PostHog token is absent, ingestion continues and events remain queued in D1 for later delivery. If the identity-link token is absent, account linking returns `503` while ordinary ingestion continues.

The identity-link token is service-to-service authority. Rotate it if it is exposed, and never embed it in website or desktop bundles:

```sh
wrangler secret put IDENTITY_LINK_TOKEN --config workers/events/wrangler.jsonc
```

After an account service has authenticated a user and obtained their opaque account, installation, or visitor IDs, an authorized operator can exercise the same service endpoint with:

```sh
SCIENT_IDENTITY_LINK_TOKEN=... bun run identity:link \
--account account:<uuid> \
--identity installation:<uuid> \
--identity visitor:<uuid>
```

This command is an operational bridge, not a substitute for account authentication. The eventual account service should call the endpoint server-to-server after sign-in; no link token belongs in a client bundle.
70 changes: 54 additions & 16 deletions functions/_lib/events.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
// FILE: events.ts
// Purpose: Stores a deliberately small, anonymous set of first-party website events.
// Purpose: Stores a deliberately small set of consent-aware first-party website events.
// Layer: Cloudflare Pages Function utility

import { cleanSessionId, websiteIdentity, websiteSessionId } from "./identity";

export const SITE_EVENT_NAMES = [
"page_viewed",
"download_clicked",
Expand All @@ -21,6 +23,7 @@ export interface SiteEvent {
readonly destinationPath?: string | null;
readonly failureStage?: string | null;
readonly failureReason?: string | null;
readonly sessionId?: string | null;
}

export interface SiteEventContext {
Expand All @@ -37,8 +40,12 @@ const INSERT_EVENT = `
privacy_level,
occurred_at,
distinct_id,
properties_json
) VALUES (?, ?, 'website', ?, ?, ?, ?)
properties_json,
identity_type,
canonical_id,
session_id,
consent_level
) VALUES (?, ?, 'website', ?, ?, ?, ?, ?, ?, ?, ?)
`;

const PRODUCTION_HOSTS = new Set(["scientfactory.com", "www.scientfactory.com"]);
Expand Down Expand Up @@ -79,8 +86,18 @@ export function shouldPersistSiteEvents(request: Request): boolean {
return PRODUCTION_HOSTS.has(new URL(request.url).hostname);
}

export async function insertSiteEvent(db: D1Database, event: SiteEvent): Promise<void> {
export async function insertSiteEvent(
db: D1Database,
event: SiteEvent,
request?: Request,
): Promise<void> {
const eventId = crypto.randomUUID();
const identity = request ? websiteIdentity(request) : null;
const distinctId = identity?.identityId ?? `web-event:${eventId}`;
const occurredAt = new Date().toISOString();
const sessionId = identity
? (websiteSessionId(request!) ?? cleanSessionId(event.sessionId))
: null;
const properties = Object.fromEntries(
Object.entries({
page_path: cleanPath(event.pagePath),
Expand All @@ -94,17 +111,38 @@ export async function insertSiteEvent(db: D1Database, event: SiteEvent): Promise
}).filter((entry): entry is [string, string] => entry[1] !== null),
);

await db
.prepare(INSERT_EVENT)
.bind(
eventId,
event.eventName,
event.eventName === "download_failed" ? "diagnostic" : "product",
new Date().toISOString(),
`web-event:${eventId}`,
JSON.stringify(properties),
)
.run();
const statements: D1PreparedStatement[] = [];
if (identity) {
statements.push(
db
.prepare(
`INSERT INTO analytics_identities (
identity_id, identity_type, canonical_id, consent_level, first_seen_at, last_seen_at
) VALUES (?, 'web_visitor', ?, 'product', ?, ?)
ON CONFLICT(identity_id) DO UPDATE SET
last_seen_at = excluded.last_seen_at,
consent_level = 'product'`,
)
.bind(identity.identityId, identity.canonicalId, occurredAt, occurredAt),
);
}
statements.push(
db
.prepare(INSERT_EVENT)
.bind(
eventId,
event.eventName,
event.eventName === "download_failed" ? "diagnostic" : "product",
occurredAt,
distinctId,
JSON.stringify(properties),
identity?.identityType ?? "event",
identity?.canonicalId ?? distinctId,
sessionId,
identity?.consentLevel ?? "essential",
),
);
await db.batch(statements);
}

export function queueSiteEvent(context: SiteEventContext, event: SiteEvent): void {
Expand All @@ -116,7 +154,7 @@ export function queueSiteEvent(context: SiteEventContext, event: SiteEvent): voi
return;
}

const write = insertSiteEvent(db, event).catch((error: unknown) => {
const write = insertSiteEvent(db, event, context.request).catch((error: unknown) => {
console.error(
JSON.stringify({
message: "Site event write failed",
Expand Down
124 changes: 124 additions & 0 deletions functions/_lib/identity.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
// FILE: identity.ts
// Purpose: Resolve consent-aware first-party website identity without browser fingerprinting.
// Layer: Cloudflare Pages Function utility

export const ANALYTICS_NOTICE_VERSION = "2026-07-identity-v1";
export const ANALYTICS_CONSENT_COOKIE = "sf_analytics";
export const ANALYTICS_VISITOR_COOKIE = "sf_visitor";
export const ANALYTICS_SESSION_COOKIE = "sf_session";

export type WebsiteConsentLevel = "essential" | "product";

export interface WebsiteIdentity {
readonly identityId: string;
readonly identityType: "web_visitor";
readonly canonicalId: string;
readonly consentLevel: "product";
}

const VISITOR_ID_PATTERN =
/^visitor:[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
const SESSION_ID_PATTERN =
/^session:[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;

function cookieMap(request: Request): ReadonlyMap<string, string> {
const entries = (request.headers.get("Cookie") ?? "")
.split(";")
.map((part) => part.trim())
.filter(Boolean)
.flatMap((part): ReadonlyArray<readonly [string, string]> => {
const separator = part.indexOf("=");
if (separator < 1) return [];
const key = part.slice(0, separator);
const value = part.slice(separator + 1);
try {
return [[key, decodeURIComponent(value)]];
} catch {
return [];
}
});
return new Map(entries);
}

export function websiteConsentLevel(request: Request): WebsiteConsentLevel | null {
const value = cookieMap(request).get(ANALYTICS_CONSENT_COOKIE);
return value === "essential" || value === "product" ? value : null;
}

export function websiteIdentity(request: Request): WebsiteIdentity | null {
if (websiteConsentLevel(request) !== "product") return null;
const identityId = cookieMap(request).get(ANALYTICS_VISITOR_COOKIE);
if (!identityId || !VISITOR_ID_PATTERN.test(identityId)) return null;
return {
identityId,
identityType: "web_visitor",
canonicalId: identityId,
consentLevel: "product",
};
}

export function cleanSessionId(value: string | null | undefined): string | null {
return value && SESSION_ID_PATTERN.test(value) ? value : null;
}

export function websiteSessionId(request: Request): string | null {
return cleanSessionId(cookieMap(request).get(ANALYTICS_SESSION_COOKIE));
}

export function newVisitorId(): string {
return `visitor:${crypto.randomUUID()}`;
}

function secureCookie(request: Request): string {
return new URL(request.url).protocol === "https:" ? "; Secure" : "";
}

export function consentCookies(
request: Request,
level: WebsiteConsentLevel,
visitorId?: string,
): ReadonlyArray<string> {
const common = `Path=/; Max-Age=${180 * 24 * 60 * 60}; SameSite=Lax${secureCookie(request)}`;
const cookies = [`${ANALYTICS_CONSENT_COOKIE}=${level}; ${common}`];
if (level === "product" && visitorId) {
cookies.push(`${ANALYTICS_VISITOR_COOKIE}=${visitorId}; ${common}; HttpOnly`);
} else {
cookies.push(
`${ANALYTICS_VISITOR_COOKIE}=; Path=/; Max-Age=0; SameSite=Lax${secureCookie(request)}; HttpOnly`,
);
}
return cookies;
}

export async function persistWebsiteConsent(
database: D1Database,
level: WebsiteConsentLevel,
recordedAt: string,
identityId: string | null,
): Promise<void> {
const statements: D1PreparedStatement[] = [];
if (identityId) {
statements.push(
database
.prepare(
`INSERT INTO analytics_identities (
identity_id, identity_type, canonical_id, consent_level, first_seen_at, last_seen_at
) VALUES (?, 'web_visitor', ?, ?, ?, ?)
ON CONFLICT(identity_id) DO UPDATE SET
consent_level = excluded.consent_level,
last_seen_at = excluded.last_seen_at`,
)
.bind(identityId, identityId, level, recordedAt, recordedAt),
);
}
statements.push(
database
.prepare(
`INSERT INTO analytics_consents (
consent_id, identity_id, source, consent_level, notice_version, recorded_at
) VALUES (?, ?, 'website', ?, ?, ?)`,
)
.bind(crypto.randomUUID(), identityId, level, ANALYTICS_NOTICE_VERSION, recordedAt),
);
await database.batch(statements);
}
95 changes: 95 additions & 0 deletions functions/api/consent.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
import { afterEach, describe, expect, it, vi } from "vitest";

import { onRequestPost } from "./consent";

function createDatabase() {
const run = vi.fn().mockResolvedValue({ success: true });
const bind = vi.fn(() => ({ run }));
const prepare = vi.fn(() => ({ bind }));
const batch = vi.fn().mockResolvedValue([]);
return { database: { prepare, batch } as unknown as D1Database, bind, prepare, batch };
}

function createContext(level: unknown, cookie?: string) {
const database = createDatabase();
const request = new Request("https://scientfactory.com/api/consent", {
method: "POST",
headers: {
"Content-Type": "application/json",
Origin: "https://scientfactory.com",
...(cookie ? { Cookie: cookie } : {}),
},
body: JSON.stringify({ level }),
});
return {
context: { request, env: { DOWNLOAD_DB: database.database } } as Parameters<
typeof onRequestPost
>[0],
database,
};
}

afterEach(() => vi.restoreAllMocks());

describe("website analytics consent", () => {
it("creates a random first-party visitor after product consent", async () => {
const { context, database } = createContext("product");
const response = await onRequestPost(context);

expect(response.status).toBe(200);
const cookies = response.headers.getSetCookie();
expect(cookies).toEqual(
expect.arrayContaining([
expect.stringMatching(/^sf_analytics=product;/),
expect.stringMatching(/^sf_visitor=visitor:[0-9a-f-]+;/),
]),
);
expect(database.batch).toHaveBeenCalledOnce();
expect(database.bind).toHaveBeenCalledWith(
expect.stringMatching(/^visitor:/),
expect.stringMatching(/^visitor:/),
"product",
expect.any(String),
expect.any(String),
);
});

it("removes persistent identity when essential-only is selected", async () => {
const { context, database } = createContext(
"essential",
"sf_analytics=product; sf_visitor=visitor:16ace444-e7c3-4b26-893f-98713188ae52",
);
const response = await onRequestPost(context);

expect(response.status).toBe(200);
expect(response.headers.getSetCookie()).toEqual(
expect.arrayContaining([
expect.stringMatching(/^sf_analytics=essential;/),
expect.stringContaining("sf_visitor=; Path=/; Max-Age=0"),
]),
);
expect(database.batch).toHaveBeenCalledOnce();
expect(database.bind).toHaveBeenCalledWith(
expect.any(String),
"visitor:16ace444-e7c3-4b26-893f-98713188ae52",
"essential",
"2026-07-identity-v1",
expect.any(String),
);
});

it("rejects a cross-origin consent submission", async () => {
const { context, database } = createContext("product");
Object.defineProperty(context, "request", {
value: new Request("https://scientfactory.com/api/consent", {
method: "POST",
headers: { "Content-Type": "application/json", Origin: "https://attacker.example" },
body: JSON.stringify({ level: "product" }),
}),
});

const response = await onRequestPost(context);
expect(response.status).toBe(403);
expect(database.prepare).not.toHaveBeenCalled();
});
});
Loading