Add admin-only /testall diagnostics with AST audit and runtime sim

AGENTS/SESSION_LOG.md

@@ -47,3 +47,52 @@
   to be updated in next session (low risk; endpoint is a companion service)
 - Modularize `index.js` — deferred pending >80% test coverage
 - Memory eviction for inactive users — deferred pending user count >10k
+
+### 2026-03-04 — Post-audit index/map synchronization
+
+**Scope:** `docs/INDEX.md`, `RUNEWAGER_FUNCTIONALITY_MAP.md`, `reports/`
+
+**Changes:**
+- Re-ran AST audit (`node scripts/ast_audit.js`) to regenerate deterministic reports after verification pass.
+- Updated `docs/INDEX.md` metadata and architecture notes with current command/action counts and audit invariants.
+- Documented intentional command normalization residual (`/A` alias normalized to `a` in group guard), fallback callback observability behavior, and current no-Scenes design status.
+- Appended map update entry to `RUNEWAGER_FUNCTIONALITY_MAP.md` future updates log to keep map/index/report agreement explicit.
+
+**Tests:** `node scripts/ast_audit.js`; `npm test` (60/60 pass)
+**Map updated:** yes
+**Index updated:** yes
+**Open items:**
+- Consider staged Scene/Stage introduction only if future flows need explicit FSM isolation; no change applied in this session.
+
+### 2026-03-04 — /testall AST + runtime simulation extension
+
+**Scope:** `index.js`, `scripts/testall_runtime.js`, `reports/`, `docs/INDEX.md`, `docs/features/10-deploy-ops.md`, `RUNEWAGER_FUNCTIONALITY_MAP.md`
+
+**Changes:**
+- Added `scripts/testall_runtime.js` deterministic Telegraf update harness to simulate all discovered commands and callback payloads, capture API-call outputs, and write JSON/Markdown reports.
+- Updated `/testall` command to run `scripts/ast_audit.js` + `scripts/testall_runtime.js` and include pass/warn/fail signals in the existing diagnostic summary.
+- Added guarded `module.exports = { bot }` path for internal simulation mode (`TESTALL_EXPORTS=1`) without changing VPS runtime behavior.
+- Updated map/index/deploy-ops docs to reflect new `/testall` report artifacts and audit workflow.
+
+**Tests:** `node scripts/ast_audit.js`; `node scripts/testall_runtime.js`; `npm test` (60/60 pass)
+**Map updated:** yes
+**Index updated:** yes
+
+## 2026-03-04 — /testall comprehensive self-contained runner hardening
+- Reworked `/testall` in `index.js` to enforce admin-only entry via `isAdmin(ctx)` and run a full internal step registry with progress updates, non-blocking step failure handling, and deterministic report generation.
+- Added ephemeral runtime snapshot + rollback helpers (`createTestAllStateSnapshot`, `restoreTestAllStateSnapshot`) covering user/promo/giveaway/tips/promo-history/smart-button/sshv runtime stores.
+- Added synthetic command/callback simulation through `bot.handleUpdate` for all `BOT_KNOWN_COMMANDS` entries and AST-discovered actions (literal + representative regex callback samples), with fallback/unmatched callback capture.
+- Added deterministic report persistence and batched admin DM summary output for `/testall` (`reports/testall_report.json`, `reports/testall_report.md`).
+- Synced documentation: `docs/INDEX.md` and `RUNEWAGER_FUNCTIONALITY_MAP.md` now reflect the expanded `/testall` behavior and rollback/report invariants.
+- Regenerated AST and runtime reports.
+
+## 2026-03-04 — /testall rotating log retention and report-linking
+- Added `rotating-file-stream` runtime dependency and integrated a dedicated `/testall` rotating logger (`data/logs/testall/testall.log*`) with daily/10MB rotation, gzip compression, and retention cap.
+- Updated `/testall` to emit step progress/errors plus command/action outcomes to rotating log files while still storing structured in-memory report results.
+- Extended final artifacts to include `reports/testall_report.txt` in addition to JSON/Markdown, and embedded rotating-log references (latest + retained files) into report payload + admin DM summary.
+- Kept admin-only enforcement, snapshot/rollback, callback fallback observability capture, and deterministic audit report regeneration behavior intact.
+
+## 2026-03-04 — CI smoke fix for rotating logger dependency
+- Removed top-level hard require for `rotating-file-stream` and moved it to lazy loading inside `createTestAllRotatingLogger`.
+- Added append-writer fallback with bounded file pruning so `/testall` logging remains available when rotation transport is missing.
+- Kept all `/testall` reporting + admin DM behavior intact and regenerated AST/runtime reports.

RUNEWAGER_FUNCTIONALITY_MAP.md

@@ -85,7 +85,7 @@ Navigation is driven by inline menus plus command aliases. Persistent user/admin
 - Helpful Tooltips manager shortcut
 
 ### Admin category menus
-- TestAll engine (`/testall`) runs structured diagnostics across environment, data/stores, callbacks/commands, navigation helpers, giveaway/promo/helpful-tooltips, SSHV, pendingAction timeout/label rules; summary line: `TestAll complete — X passed, Y warnings, Z failures.`
+- TestAll engine (`/testall`) is admin-only and now behaves as a comprehensive self-contained runner: captures ephemeral snapshots of runtime stores, executes AST audit + full command/action simulation via synthetic updates, includes pagination/admin-flow/FSM-scene status checks, continues on step failure, records fallback/unmatched callbacks, restores snapshot rollback state, DMs batched final report to all admins, and writes deterministic artifacts to `reports/testall_report.json` + `reports/testall_report.md` + `reports/testall_report.txt`; each step also streams verbose progress/error logs to rotating files under `data/logs/testall/` (daily or 10MB, gzip compressed, retention window), and invariant summary line remains: `TestAll complete — X passed, Y warnings, Z failures.`
 - `admin_cat_giveaway`: start/test/status + persistent navigation row (`Admin Dashboard`, `Main Menu`, `Cancel`).
 - `admin_cat_promo`: full promo manager actions + guide + persistent navigation row (`Admin Dashboard`, `Main Menu`, `Cancel`).
 - `admin_cat_system`: health/version/verify/setup/backup/admin mode/testall/sshv + **🔗 Group Linking** (v3.1 — opens group linking tools with return to System Tools) + persistent navigation row (`Admin Dashboard`, `Main Menu`, `Cancel`).
@@ -410,6 +410,8 @@ Mandatory rules for any AI agent touching this repo:
 - 2026-02-28: v3.1 — added group command guard middleware (`GROUP_PASSTHROUGH_COMMANDS` + `bot.use` interceptor); added `onboardingProgressBar()` and progress header on each onboarding step prompt (auto-deletes after 8s); added one-time onboarding completion card (tracked via `user.onboarding.completionCardShown`); added `🔗 Group Linking` to Admin System Tools keyboard (`admin_sys_group_linking` callback with back-to-system-tools navigation).
 - 2026-03-04: Merged main branch additions — `telegramSafe.js` (global rate-limit patch via `telegramSafe.init(bot)`), `rateLimiter.js` (per-chat + global Telegram API rate limiter), `backend.js` (companion HTTP service on port 3001, `runewager-endpoint.service` systemd unit), updated `prod-run.sh`, `scripts/runewager_redeploy.sh`, `scripts/rw_cpu_guard.sh`.
 - 2026-03-04: /runewager-audit — 17-phase full audit. Findings: 0 critical, 1 warning resolved (dead `adminKeyboard()` function removed — legacy promo keyboard with no callers). Auto-fix applied. 60/60 tests pass post-fix.
+- 2026-03-04: Post-audit alignment verification pass — re-ran `node scripts/ast_audit.js` and regenerated `reports/ast_audit.json` + `reports/structural_audit.md`; confirmed command normalization residual is intentional (`/A` handled through lowercase `a` in group guard), fallback callback handler still preserves user-facing recovery while logging bounded unmatched callback context, and no Scene/Stage FSM wiring is currently present by design.
+- 2026-03-04: Extended `/testall` to execute AST audit + deterministic internal update simulation (all discovered commands/callbacks), writing `reports/testall_report.json` and `reports/testall_report.md` while preserving existing admin-only behavior and summary format.
 - 2026-03-01: Created `docs/` feature documentation system — 15 per-feature `.md` files, central `docs/INDEX.md` with full callback + pending-action cross-reference, and `docs/TODO_FUNCTIONALITY_UPGRADE.md` tracking 14 open upgrade/stale-menu items. Future Claude sessions must consult `docs/INDEX.md` first, then the relevant feature `.md`, before reading `index.js`.
 - 2026-03-01: Phase implementation — resolved T-01/T-02/T-03/T-15 from TODO list. (1) Walkthrough: `sendWalkthroughStep()` upgraded with `clearOldMenus()`, Back disabled on step 1, Finish on last step, `walk_done` on last step returns to main menu. New doc: `16-walkthrough.md`. (2) Menu stacking: `clearOldMenus()` added to `sendOnboardingReferralPrompt`, `renderSshvConsole`, `renderGroupLinkingTools`, `tips_cmd_edit`, `tips_cmd_remove`. (3) Tooltip view: `tips_cmd_view` selector + `tip_view_{id}` handler with Prev/Next/Edit/Toggle/Delete/Back/AdminMenu — "👁 View Tooltip" button added to dashboard. (4) Broadcast failures: 500-item cap removed; all failures logged via `adminLog()`; `/broadcast_failed` shows chunks of 30 with overflow note; >20% failure rate triggers admin DM warnings. PR comments fixed: `add_tooltip.sh` array validation hardened; `docs/12-group-linking.md` entry-point callback corrected to `admin_sys_group_linking`; merge conflicts (PRs #112-114) resolved keeping SIGTERM→SIGKILL safety improvements. 60/60 tests pass.
 - 2026-02-28: PR #112 review + audit pass — fixed 10 issues: (R1) `await_tip_import_batch` dedicated pending type with JSON-array router; (R2) `generate_tooltips.sh` command-substitution pollution fixed via `RUNEWAGER_APP` env var; (R3) `add_tooltip.sh` shell-injection fixed via `TOOLTIP_TEXT_ENV`/`TOOLTIP_TMP_FILE` env vars and `<<'EOF'`; (R4) `catchAllCases` test extended with multiline patterns + `CATCH_ALL_CORES` updated; (R5) `extractCommandHandlerNames` test extended with `let`/`var`/no-semicolon fixtures; (R6) typo "auto-deletes 8s" → "auto-deletes after 8s"; (A1) dead `buildGiveawayAnnouncementText(giveaway,remainingStr)` removed; (A2) simplified `buildGiveawayAnnouncementKeyboard` with wrong callback removed; (A3+A4) duplicate `bot.action('admin_cat_system')` and `bot.action('admin_cat_support')` first registrations removed. All 60 tests pass.
@@ -444,3 +446,9 @@ All bot functionality is documented in `docs/`:
 | [`docs/TODO_FUNCTIONALITY_UPGRADE.md`](docs/TODO_FUNCTIONALITY_UPGRADE.md) | T-01–T-15 upgrade log (T-01/02/03/15 resolved) |
 
 **Mandate:** Any added/changed/removed feature → update the relevant feature `.md` + `docs/INDEX.md` + this map section, in the same commit.
+
+- 2026-03-04: Upgraded `/testall` to an in-process comprehensive runner with snapshot/rollback, synthetic command+callback execution, fallback callback logging capture, and batched DM reporting to all ADMIN_IDS.
+
+- 2026-03-04: Added smart rotating log retention for `/testall` verbose output (`data/logs/testall/testall.log*`), with compressed archives and retained history references embedded in final reports + admin DM summary.
+
+- 2026-03-04: Hardened `/testall` logger bootstrap by lazy-loading rotation transport with a non-crashing append fallback; CI smoke now boots even when rotation dependency is absent, while `/testall` retains verbose log artifact output.

docs/INDEX.md

@@ -2,8 +2,8 @@
 
 > **Purpose:** This is the primary navigation index for all Claude sessions. Before modifying any feature, read the relevant `.md` file here. After any change, update both the feature `.md` and this index.
 
-**Last updated:** 2026-03-01
-**Bot version:** 3.0.0 | `index.js`: ~15,050 lines | Commands: 95 | Action handlers: 270+
+**Last updated:** 2026-03-04
+**Bot version:** 3.0.0 | `index.js`: ~15,100 lines | Commands: 96 registered (`95` normalized group-command entries) | Action handlers: 268
 
 ---
 
@@ -178,6 +178,15 @@
 - **User mutations:** `runUserMutation(userId, fn)` — queue prevents race conditions.
 - **Error handling:** `bot.catch()` global handler + `uncaughtException`/`unhandledRejection` process handlers.
 
+### Command + callback audit invariants (2026-03-04)
+
+- `BOT_KNOWN_COMMANDS` is synced with handler coverage for all normalized command names.
+- Intentional residual: `/A` is registered as an alias command, but group parsing lowercases all command names before membership checks; therefore `a` is the canonical `BOT_KNOWN_COMMANDS` entry and covers `/A` in group context.
+- Catch-all callback fallback (`bot.action(/.*/, ...)`) remains in place for UX recovery and now logs unmatched callback context (`userId` + bounded `callbackData`) for observability.
+- Current AST audit baseline: 520 callback emitters, 268 action handlers, 0 dead literal callbacks, 0 fallback-only literal callbacks, 0 literal overlap/shadow conflicts.
+- No Telegraf Scene/Stage FSM wiring is currently used (`scene.enter`/`scene.leave`/`Stage` registrations absent by design in this revision).
+- `/testall` now runs a comprehensive admin-only internal runner: AST audit, command simulation for every `BOT_KNOWN_COMMANDS` entry, action/callback simulation (including regex samples and fallback visibility), pagination/admin-flow checks, ephemeral state snapshot+restore rollback, and report persistence to `reports/testall_report.json` + `reports/testall_report.md` + `reports/testall_report.txt`; verbose per-step logs stream to rotating files in `data/logs/testall/` (daily/10MB, gzip, retained history).
+
 ---
 
 ## Known Issues → See TODO_FUNCTIONALITY_UPGRADE.md

docs/features/10-deploy-ops.md

@@ -72,14 +72,17 @@ TestAll complete — X passed, Y warnings, Z failures.
 ```
 
 Checks include:
-- Bot token validity
-- Admin ID configuration
-- Group/channel link status
-- Health endpoint reachability
-- Giveaway state consistency
-- Promo store integrity
-- Tooltip store integrity
-- Environment variable completeness
+- Existing internal diagnostics (env, stores, command/callback guards, navigation helpers, giveaways/promos/tooltips, SSHV checks).
+- Full AST audit execution via `scripts/ast_audit.js` (commands/actions/callback emitters, middleware, scene/FSM status, async/error patterns).
+- Programmatic command + callback simulation via `scripts/testall_runtime.js` using a deterministic internal Telegraf update harness (no external API calls).
+
+Artifacts written each `/testall` run:
+- `reports/ast_audit.json`
+- `reports/structural_audit.md`
+- `reports/testall_report.json`
+- `reports/testall_report.md`
+- `reports/testall_report.txt`
+- Rotating verbose logs: `data/logs/testall/testall.log*` (daily/10MB, gzip, retention cap).
 
 ---
 
@@ -133,3 +136,12 @@ Key metrics exported:
 - `index.js`: `/testall` ~13448, `admin_cmd_*` ~8940–9045
 - `deploy.sh`, `prod-run.sh`, `scripts/rollback.sh`
 - `.github/workflows/deploy.yml`, `ci.yml`
+
+
+### Rotating log policy for `/testall`
+
+- `/testall` writes per-step progress and errors to rotating file stream: `data/logs/testall/testall.log`.
+- Rotation triggers: daily interval **or** size threshold (`10M`).
+- Rotated files are gzip compressed and old files are pruned automatically (`maxFiles` retention).
+- If rotation transport is unavailable at runtime, `/testall` gracefully falls back to append logging at `data/logs/testall/testall.log` (no boot-time crash).
+- Final `/testall` reports include references to latest + retained rotating log artifacts for audit traceability.