Skip to content

Add incremental slack render Markdown projection #2

Description

@dremnik

Problem

Repeatedly reading Slack through live API calls is slow and awkward for local research. There is no filesystem-native representation that works with rg, Zed, Codex, backups, and ordinary document links.

Decision

Add slack render as an incremental projection of selected Slack conversations into deterministic Markdown. It is a current local view, not an unrestricted workspace dump or a compliance archive.

The detailed design note is docs/.ideas/slack-render.md.

Proposed surface

slack render ./workspace-slack --channel "#product" --channel "#engineering" --since 90d
slack render ./workspace-slack --update
slack render ./workspace-slack --all-channels --since 30d

Initial options:

  • --channel <ref> repeatable
  • --all-channels
  • --since <time>
  • --include-dms
  • --reconcile-days <n>
  • --prune
  • --allow-vcs
  • --config <path>

Initial scope

  • Render explicitly selected public/private channels visible to the active user profile.
  • Store one directory per durable channel ID and shard transcripts by month.
  • Embed thread replies beneath their root messages.
  • Preserve Slack timestamps, user/channel IDs, edit timestamps, reactions, file metadata, and permalinks.
  • Resolve mentions for readability while retaining source IDs.
  • Maintain machine-owned incremental state in .slack-render.json.
  • Re-fetch a rolling reconciliation window to catch recent edits and deletions.
  • Rewrite only changed monthly shards and indexes.
  • Link files by default; defer file downloads.
  • Produce no filesystem changes when Slack has not changed.

Safety defaults

  • Require explicit channels, --all-channels, or a configuration file.
  • Exclude DMs and group DMs unless --include-dms is explicit.
  • Refuse output inside a version-controlled tree unless --allow-vcs is explicit.
  • Create output directories as 0700 and files as 0600 on Unix.
  • Generate an output-local .gitignore that ignores rendered contents.
  • Never log message bodies, tokens, cookies, or signed file URLs.
  • Use the supported user-token path for bulk rendering; reject browser-session credentials.
  • Make deletion an explicit --prune operation with a preview.

Blockers

Do not begin the full implementation until:

  • pagination returns correct boundaries and continuation state when --limit is smaller than a Slack response page;
  • user identity and credential-handling findings from the implementation review are resolved;
  • the Web API base URL and transport are injectable for tests;
  • mock coverage exists for ok:false, 429, malformed responses, pagination, and timeouts.

First spike

Render one selected channel over a bounded date range. The fixture must contain a root message, edited message, thread, reaction, mention, and file link.

The spike succeeds when:

  1. The Markdown is readable and contains durable source metadata.
  2. A second run with no Slack changes produces no diff.
  3. Adding and editing fixture messages rewrites only the affected monthly shard.
  4. Rate limits resume without corrupting state or leaving partial output.

Acceptance criteria

  • slack render <output> --channel <ref> --since <time> creates the documented directory structure and deterministic Markdown.
  • slack render <output> --update fetches only required history plus the reconciliation window.
  • Threads, edits, reactions, mentions, file links, and permalinks render correctly.
  • DMs, file downloads, VCS output, and pruning require explicit flags.
  • Interrupted runs do not advance state or replace valid files with partial output.
  • No-op updates leave the output tree byte-for-byte unchanged.
  • Unit and mock integration tests cover pagination, reconciliation, atomic writes, unsafe output paths, and rate limiting.

Out of scope

  • Real-time synchronization through Events API or Socket Mode.
  • Compliance exports, legal holds, or archival-completeness claims.
  • HTML, PDF, and static-site output.
  • Search indexes beyond filesystem text search.
  • Downloading file contents in the first implementation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions