Skip to content

feat: add package wrapper install telemetry #260

@parsakhaz

Description

@parsakhaz

Intent

Add first-class telemetry for the runpane npm and PyPI wrappers so we can see the full package-wrapper funnel, not just the final Pane artifact download. This matters because download_started with source=npm or source=pip proves a wrapper requested a release artifact, but it does not show whether someone installed the package, invoked the wrapper, chose client vs daemon setup, succeeded, failed, or fell back to GitHub.

Scope

Track wrapper-level activity for both package surfaces:

  • npm ecosystem: npm, npx, pnpm dlx, yarn dlx, bunx, global npm/pnpm installs where detectable.
  • Python ecosystem: pip, pipx, uvx, and python -m runpane where detectable.
  • Commands that should be distinguishable: setup, install client, install daemon, and update.
  • Outcomes that should be distinguishable: command started, artifact download requested, download succeeded, download failed, install/setup succeeded, install/setup failed, and fallback-to-GitHub used.
  • Preserve the current download_started source attribution (source=npm / source=pip) for the actual artifact redirect.

Starting Points

These references are non-exhaustive; investigate further before implementation.

  • packages/runpane/src/cli.ts
  • packages/runpane/src/releases.ts
  • packages/runpane-py/src/runpane/cli.py
  • packages/runpane-py/src/runpane/releases.py
  • docs/RUNPANE_CLI_CONTRACT.md
  • runpane website docs: docs/posthog-event-catalog.md and docs/analytics-attribution.md now document package downloads as download_started segmented by properties.source.

Acceptance Criteria

  • We can answer, in PostHog, how many users invoked the npm wrapper versus the PyPI wrapper over a selected time window.
  • We can distinguish npx/npm-family usage from pipx/pip/uvx-style usage when the runtime makes that reasonably detectable.
  • We can distinguish client install, daemon install, setup, and update wrapper commands.
  • We can see started, succeeded, failed, and fallback outcomes without relying on raw logs.
  • Events include non-sensitive context such as wrapper, wrapper_version, invocation, command, platform, arch, pane_version, and download_source.
  • Events must not include local paths, usernames, hostnames, tokens, connection codes, raw errors, full URLs with sensitive query params, or environment dumps.
  • Existing /api/download telemetry still emits download_started with source=npm or source=pip when the wrapper requests a Pane artifact.
  • The PostHog event catalog and analytics attribution docs are updated with the final event names and properties.
  • Tests cover the telemetry property builder/sanitizer for npm and Python paths, including failure/fallback cases.

Inputs Needed

Decide whether wrapper telemetry should be sent directly from the package clients to PostHog/runpane.com, or whether it should go through a new runpane.com API route that validates and forwards sanitized events. Prefer a server route if direct client capture would expose keys or make schema evolution harder.

Notes

Current observed data shows package-wrapper artifact downloads are present via download_started: source=npm and source=pip. That is useful but incomplete: it only records artifact fetches through /api/download, not package install, wrapper invocation, command intent, success/failure, or GitHub fallback.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions