Decide on a privacy-preserving usage analytics strategy for ExtractMD

[miguelcorderocollar]

Summary

We should use two layers of analytics:

1. Chrome Web Store / Developer Dashboard metrics for distribution and adoption
2. Minimal, privacy-preserving in-product telemetry for feature usage inside the extension

Recommendation:

• Use Chrome Web Store analytics natively for installs, uninstalls, listing impressions, and weekly users.
• Add opt-in anonymous product analytics inside the extension to answer questions the Web Store cannot answer, such as:
  • how many people actually use ExtractMD after install
  • which integrations are used most (youtube, hackernews, article, future integrations)
  • which entry points are used most (popup, keyboard shortcut, floating button, context menu if added later)
  • whether users hit extraction failures or empty outputs
  • whether users open and actively use settings
  • whether users ignore/hide the extension on pages or domains
  • which optional features are adopted versus left untouched
• Prefer PostHog with manual event capture only over GA4 or Plausible for extension telemetry.
• If privacy requirements become stricter later, use PostHog EU Cloud first, and consider self-hosted PostHog only if we have a real operational or compliance reason.

Why this is the right split

What Chrome Web Store gives us natively

Chrome Web Store analytics already covers important top-of-funnel questions:

• installs and uninstalls
• store listing impressions
• weekly users
• breakdowns by country, language, operating system, and item version

Important limitation: Chrome’s own docs say the Users stats only captures installations; it does not monitor whether users are active or not. In other words, Web Store analytics can tell us about distribution, but not real feature usage inside the extension.

That means Chrome Web Store analytics cannot answer:

• how often the popup is opened
• how many extractions succeed or fail
• which integration users actually extract from most
• whether users prefer the keyboard shortcut vs popup vs floating button
• whether users open settings but never change anything
• which settings are actually used in practice
• whether users ignore pages or disable the tool on domains
• whether a given release improves retention in-product

Why not rely on Google Analytics alone

Google has official guidance for tracking extension usage with GA4 Measurement Protocol under Manifest V3, so it is a valid option.

However, for this use case, GA4 is not the best primary choice:

• it is better suited to general analytics than product analytics decision-making
• event exploration and product-style analysis are weaker for questions like “which integration is most used?” or “which settings are actually adopted?”
• it is not the privacy-first option we likely want for an extension that touches page content
• the Chrome Web Store GA integration is for the store listing, not for detailed in-extension product analytics

Conclusion: GA4 is acceptable, but not the best default for ExtractMD.

Why PostHog is the best fit

PostHog is the strongest fit if we want actionable product analytics with tight privacy controls.

Benefits:

• event-first product analytics is a better match for extension usage questions
• supports custom events and properties cleanly
• supports explicit opt-in / opt-out controls
• supports disabling autocapture entirely
• supports filtering or masking sensitive data before it is sent
• can be run in EU Cloud or self-hosted if needed later

Most important best-practice decision: do not enable autocapture.

For ExtractMD, autocapture would be the wrong default because the extension operates around page content and user interactions. We should only send a small hand-curated set of manual events with synthetic properties such as:

• integration name
• trigger source
• extension version
• success / failure
• error category
• output size bucket
• settings area or setting key
• ignore action scope (page or domain)

We should never send:

• page content
• transcript/article text
• URLs
• page titles
• clipboard contents
• account identifiers
• ignored domain names in raw form
• free-form error strings that may contain page data

Why Plausible is not the best primary choice

Plausible is attractive because it is privacy-friendly, cookie-free, and simple. It is a good option when the main goal is lightweight aggregate counts.

But it is weaker for this exact problem:

• it is primarily web analytics, not product analytics
• richer segmentation depends on custom properties
• it is less natural for event-heavy extension analysis than PostHog
• it is a better fit for website traffic than extension behavior

Conclusion: Plausible is a reasonable low-complexity alternative, but not the best long-term fit if we want to learn which product flows users actually use.

Recommended implementation approach

Phase 1: Use native Chrome Web Store analytics immediately

Track and review:

• installs / uninstalls
• listing impressions
• weekly users
• version breakdowns

This gives us a baseline without changing the extension.

Phase 2: Add optional anonymous telemetry inside the extension

Add a setting like:

• Help improve ExtractMD with anonymous usage analytics

Recommended default:

• off by default for the first release of telemetry

Reason:

• the current privacy policy explicitly says ExtractMD does not use analytics or tracking services
• the extension has broad site access (<all_urls>), so user trust matters more than squeezing out slightly more data volume
• opt-in telemetry is easier to defend ethically and in store review if the implementation is minimal and clearly disclosed

If we later decide to switch to default-on, that should happen only after:

• the event schema is proven minimal
• the privacy policy is updated
• the disclosure copy is reviewed carefully
• we confirm the policy declarations in the Chrome Web Store dashboard are accurate

Phase 3: Use a strict event schema

Recommended initial events:

• extension_opened
  • properties: surface (popup, options)
• extraction_requested
  • properties: integration, trigger_source, extension_version
• extraction_completed
  • properties: integration, trigger_source, success, output_bucket, duration_bucket
• extraction_failed
  • properties: integration, trigger_source, error_code
• settings_opened
  • properties: surface (popup, options)
• settings_changed
  • properties: setting_key, setting_group, change_type
• settings_reset
  • properties: setting_group
• ignore_action_used
  • properties: scope (page, domain), action (hide, show, add_ignore, remove_ignore), source (shortcut, options, popup, button)
• floating_button_toggled
  • properties: enabled
• feature_used
  • properties: feature_name

For the first implementation, the most valuable properties are likely:

• integration
• trigger_source
• setting_key
• setting_group
• scope
• action
• extension_version
• success
• error_code

Schema rules:

• only enumerated properties
• no raw text payloads
• no URLs or domains
• no page metadata
• no persistent user identity unless there is a strong reason
• prefer anonymous device/session identifier managed by the analytics tool

Interpretation examples:

• settings_opened without settings_changed tells us people inspect settings but do not adopt them
• ignore_action_used tells us whether ignore/hide controls solve noise or friction
• settings_changed by setting_key tells us which knobs are actually valuable
• feature_used helps us avoid keeping UI/features that nobody uses

Phase 4: Update policy and disclosures before shipping

Before enabling any external telemetry:

• update docs/privacy-policy.md
• update store privacy declarations in the Chrome Web Store dashboard
• explain exactly what is collected, why, and what is never collected
• provide a clear in-product toggle and a way to disable future collection

This is mandatory because the current privacy policy says:

• no data is sent to external servers
• no analytics or tracking services are used
• no data is shared with third parties

Alternatives considered

Option A: Chrome Web Store analytics only

Pros:

• zero implementation work
• native and low risk
• no privacy policy change for extension behavior

Cons:

• does not answer product usage questions
• cannot show most-used integrations
• cannot show success/failure or UX friction
• cannot show whether settings and ignore controls are actually used

Verdict:

• necessary, but insufficient on its own

Option B: Chrome Web Store analytics + GA4 for extension events

Pros:

• officially documented by Chrome
• common and inexpensive

Cons:

• weaker product analytics workflow than PostHog
• less privacy-friendly perception
• not the cleanest fit for event/property analysis in an extension

Verdict:

• acceptable fallback if we want the most conventional path

Option C: Chrome Web Store analytics + Plausible for extension events

Pros:

• privacy-friendly
• simple mental model
• cookieless by design

Cons:

• more limited as product analytics
• less flexible for event/property exploration
• stronger fit for website analytics than extension behavior

Verdict:

• good lightweight alternative, but not the best fit

Option D: Chrome Web Store analytics + PostHog manual events

Pros:

• strongest fit for product questions
• privacy controls are explicit
• can disable autocapture completely
• can support EU hosting or self-host later

Cons:

• more implementation work than store-only
• requires privacy-policy and disclosure updates
• still introduces an external analytics dependency

Verdict:

• recommended

Proposed decision

Adopt this policy:

1. Use Chrome Web Store analytics as the default source for marketplace and adoption metrics.
2. Add opt-in anonymous telemetry for in-product usage metrics.
3. Use PostHog as the telemetry backend.
4. Send manual events only. Keep autocapture disabled.
5. Include events for extraction usage, settings adoption, and ignore/hide behavior.
6. Do not send content, URLs, titles, clipboard data, raw domains, or free-form text.
7. Update privacy policy and Chrome Web Store privacy declarations before release.

Sources

Chrome / Google

• Chrome Web Store metrics docs: https://developer.chrome.com/docs/webstore/metrics
• Chrome Web Store + GA docs: https://developer.chrome.com/docs/webstore/google-analytics
• Chrome extension GA4 guide (Manifest V3 / Measurement Protocol): https://developer.chrome.com/docs/extensions/how-to/integrate/google-analytics-4
• Chrome Web Store User Data FAQ: https://developer.chrome.com/docs/webstore/program-policies/user-data-faq
• Chrome Web Store Limited Use policy: https://developer.chrome.com/docs/webstore/program-policies/limited-use
• Chrome Web Store Privacy Policies policy: https://developer.chrome.com/docs/webstore/program-policies/privacy

PostHog

• JS usage / custom capture: https://posthog.com/docs/libraries/js/usage
• Data collection controls: https://posthog.com/docs/privacy/data-collection
• Self-hosting: https://posthog.com/docs/self-host

Plausible

• Docs overview: https://plausible.io/docs/
• Data policy: https://plausible.io/data-policy
• Events API: https://plausible.io/docs/events-api
• Custom events: https://plausible.io/docs/custom-event-goals
• Security / compliance overview: https://plausible.io/docs/compliance

Next steps

• Decide whether telemetry should be opt-in only, or ever move to default-on later
• Define the minimal event schema
• Decide whether to use PostHog EU Cloud or self-hosted PostHog
• Update privacy policy and store disclosures before implementation
• Implement telemetry behind a settings toggle
• Add tests proving forbidden fields are never sent