Add fm_analyze_pcap and fm_list_pcaps: read-only SIP capture decoder with evidence-scoped analysis

[kierknoby]

What this is

Tools/PcapAnalysis.php is a dependency-free, pure-PHP decoder for classic libpcap captures. It reads a .pcap, decodes the link, IP, and transport layers itself (no libpcap, no shell-out to tshark), lifts SIP messages, groups them by Call-ID, and produces a structured per-call analysis: outcome, final status, observations, confidence-scored diagnostic hints, and a lightweight RTP/RTCP view. Tools/ListPcaps.php is a companion picker that lists available captures newest-first as clickable chips.

fm_analyze_pcap is the primary capability introduced by this PR. fm_list_pcaps is a small companion discovery tool that lists captures from the allowlisted capture directories and emits analyse actions. It exists to make the analysis workflow usable from chat without requiring users to know capture paths.

Both are read-only and PERM_ADMIN. Neither starts, stops, nor configures a capture; fm_list_pcaps directs users to the Sysadmin Packet Capture page for capture creation and management.

From chat the workflow is list pcaps to see available captures as clickable chips, then analyse pcap <path> (or analyze, both parse) to decode one; from a result, the focus picker and the Simplify / Explain / Evidence actions take it from there. A capture path can also be passed directly without listing first.

The recommended way to produce captures is the onboard Sysadmin Pro Packet Capture feature, which writes them into the allowlisted directory. Beyond that, the decoder reads classic libpcap files from the allowlisted directories regardless of how they got there, so a capture placed in /var/spool/asterisk/packetcapture by other means is analysed under the same path, extension, and size checks; this has been exercised with classic-pcap captures from FreePBX 16 as well as 17.

Auth headers are deliberately never surfaced in output. Confirmed on real captures: the parser only retains Call-ID, CSeq, From, To, Reason, and content-type/length headers, so WWW-Authenticate and Authorization are never lifted, and captures containing 401 responses on both INVITE and OPTIONS rendered the status line only with no credential header content.

No LLM, no new database tables or schema change, no session state. A file goes in, structured analysis comes out.

Why

When a call goes wrong the capture is the ground truth, but reading a raw .pcap by hand or round-tripping through Wireshark is slow and does not belong in a chat console. This decodes the capture in-process and presents the SIP ladder the way an engineer reads it: grouped by call, ordered by time, with the final status and any Reason header foregrounded.

The principle throughout is that false confidence is worse than incomplete analysis. The tool states what the packets show as fact, offers interpretation as explicitly labelled and confidence-scored additions, and never asserts a cause the capture cannot support. Absence of RTP is reported as "no RTP seen at this capture point", never "no audio". One-way RTP is "media visibility asymmetry", never "one-way audio". It reasons like an engineer who is careful to separate what they saw from what they infer.

Architecture

Pure-PHP decode path: classic pcap magic (LE/BE, micro and nano timestamps); Ethernet, Linux SLL, SLL2, and raw link types; IPv4 and IPv6 including extension-header walking; UDP, and TCP with sequence-ordered reassembly. SIP is lifted from UDP payloads and reassembled TCP streams, compact header forms (i, f, t, m, v, c, l) are canonicalised, and messages are grouped by Call-ID.

Per call, the analysis derives the methods seen, the final and INVITE-final status, the outcome (answered, busy, cancelled, failed, auth_challenge, incomplete, and so on), observations, and confidence-scored diagnostic hints. Byte-identical non-provisional retransmissions are detected by fingerprint, deliberately excluding provisional 183 resends and auth re-INVITEs so normal flows are not flagged as loss.

RTP/RTCP is observed with streaming counters only, with no packet retention. Streams correlate to calls by SDP media port, then by IP compatibility, then by timing overlap with the signalling window; divergence lowers confidence rather than forcing a match. Per-direction packet counts, SSRC, sequence range, and a loss estimate are reported, but the loss estimate is suppressed to null when a 16-bit sequence wrap or reorder is seen, because the gap arithmetic is then unreliable. RTCP (PT 200 to 204) is detected separately and never mixed into RTP loss. Codec mismatch against SDP is low-confidence only, since dynamic payload types are SDP-scoped.

Follow-up views (Simplify, Explain, Evidence) re-run the tool with summary_action, section, and item_id parameters rather than caching analysis in session state, so there is no stored state to go stale.

In chat, a multi-transaction capture presents a focus picker: the top candidates are grouped into "Calls found" (INVITE call flows) and "Other SIP transactions" (OPTIONS qualifies, auth challenges, and the like), each row tappable to re-run the tool scoped to that one Call-ID. The grouping makes the INVITE-vs-OPTIONS split visible at a glance, so a reader can tell a real call attempt from a keepalive without reading every ladder, which matters because a busy box's capture is mostly qualifies. Scoping by Call-ID collapses the output to the single ladder of interest.

Where a capture contains two cancelled INVITE legs that look like the same propagated cancellation (overlapping time windows, opposite PBX-side direction sharing a common endpoint, both showing a 183, and cancellation evidence within a tight 250ms window), the support summary surfaces a "possible related call legs" line. This is offered at medium confidence and explicitly worded as consistent with the PBX propagating cancellation between bridged or forwarded legs, while stating the capture does not prove the application-level reason. It is gated by four independent checks so unrelated near-simultaneous cancels are not paired.

Both the focus picker rows and the call-scoped follow-up focus-context block carry a plain-English fact line, for example a cancelled INVITE reads as the From party trying to call the To party at a given time and being cancelled before answer. The verb tracks the decoded outcome, so a cancelled or failed attempt is never described as a connected call and talk time is claimed only for answered calls. Caller and callee are read from the From and To URIs and shown as the signalling carried them: a numeric user part is shown as the number, a non-numeric user part (an SBC username, for instance) degrades to the URI host, and a URI with neither degrades to a neutral label. The host is printed verbatim as a fact about the signalling, without asserting whether the endpoint is internal or external or ascribing it a PBX or trunk role. The line is composed in the tool (so the outcome-verb discipline lives in one place) and the formatter only renders it; non-INVITE transactions stay silent.

When a call is one leg of a detected related-legs pair, the picker line is framed by direction rather than by who dialled whom: it reads "Outbound leg from X to Y, cancelled..." or "Inbound leg from X to Y, cancelled...". This avoids a specific misread. On a PBX that forwards an inbound call back out, the outbound leg's From header carries the original caller's number, so the bare "X tried to call Y" wording implies that caller personally dialled the forwarded destination, which the capture does not show. The leg-framed wording states the leg's direction (proven from packet source and destination against the shared PBX endpoint) and its From and To as the headers carried them, without asserting that the call was forwarded; the inter-leg relationship stays only in the hedged related-legs line, which the capture cannot prove beyond consistency. The direction-framed wording applies only to calls in a detected pair; a standalone INVITE keeps the plain "X tried to call Y" form, which is correct for a genuine single origination. As a deliberate consequence, the call-scoped focus-context line (shown when one Call-ID is explicitly opened) keeps the plain wording even for a leg that the picker frames by direction, because in that view the leg is shown alone rather than beside its sibling, so "this leg as its headers read" is the honest framing. Plumbing direction into the focus context is a separate, larger change kept out of scope here.

The response-level Simplify action carries a plain-English one-line gloss below its existing text, introduced as a separate sentence prefixed "This means:". The existing Simplify text is accurate but reads in engineer register; the gloss restates the single salient finding for a non-technical reader, for example "at least one call was answered, and we saw audio going both ways at this capture point" or "the call ended before it was answered". It is a fixed hand-written lookup, never generated, and is additive: the technical Simplify text and the Explain and Evidence narratives are untouched, so Explain keeps the fuller register and only Simplify gains the gloss. The gloss derives from the same salient-finding selector that chooses the headline sentence, so the two cannot drift; for an answered call the gloss additionally reads the answered RTP status to say whether audio was seen both ways, one way, or not at the capture point, which enriches the explanation of the answered finding without changing which finding the selector picks. The wording holds the same evidence discipline as the rest of the tool: media is only mentioned as seen at the capture point and never as proof of what either party heard, a benign signalling gap is not surfaced as the headline on a completed call, a no-INVITE capture is described as background traffic rather than asserting no call existed, and cancellation stays neutral on who ended the call.

Safety and evidence discipline

Path resolution is allowlisted to the two capture directories (/var/spool/asterisk/frogman/captures and /var/spool/asterisk/packetcapture), realpath-checked, extension-checked, size-capped at 200 MiB, with per-packet and total TCP-reassembly byte caps to bound memory on a malformed or hostile file. Failed non-INVITE transactions are described without assuming an INVITE flow (a 403 to an OPTIONS qualify reads "SIP transaction ended in a failure response", not "INVITE ended..."). Every interpolated user-controlled field in the chat formatter passes through sanitizeForChat() and is backtick-wrapped, per the GHSA-7qvv formatter-breakout class.

Capture paths feed clickable chat command chips, so a path containing a command delimiter or control character (|, {{, }}, control bytes) is treated as unsafe to render as a chip: those rows fall back to showing the filename and path as plain backtick-wrapped text with no clickable command emitted, rather than risk a malformed or misleading chip. A normal Sysadmin capture path renders clickable as usual. This was exercised with a deliberately adversarial filename (test|evil.pcap): the file and its path stayed visible, no command chip was generated.

Test plan

Verified on a FreePBX 17 box against real Sysadmin captures via the chat console.

• Two real answered-call captures were decoded end to end. sysadmin_2025_05_22_16_56_32.pcap decoded to 17 SIP messages across 6 SIP transactions, of which 1 was an INVITE call flow answered with a clean INVITE, 100, 183, 200, ACK, BYE (cause=16) sequence, SDP lifted both ways, RTP seen in both directions, and the remaining transactions classified as OPTIONS qualifies. sysadmin_2025_05_22_14_52_43.pcap decoded to 17 SIP messages across 5 SIP transactions with the same single answered INVITE plus a 404 OPTIONS, a 501 NOTIFY, and 1 unparsed SIP-like payload. Both confirmed that an answered INVITE reconstructs correctly with SDP and bidirectional RTP, that non-INVITE failures (404, 501) are described as SIP-transaction failures and explicitly de-contaminated from the call verdict in the reader summary, and that the unparsed-payload path renders its incomplete-totals caveat correctly.
• A real cancel-timer capture (sysadmin_2026_06_01_11_20_42.pcap) decoded to 33 SIP messages across 8 SIP transactions, of which 2 were INVITE call flows, with both ~26-second cancel-before-answer flows reconstructed, and the OPTIONS/403 transaction correctly described as a SIP-transaction failure rather than an INVITE failure. The chat focus picker grouped the two cancelled INVITEs under "Calls found" and the OPTIONS failure under "Other SIP transactions"; tapping a row re-ran the tool scoped to that Call-ID and collapsed the output to the single ladder.
• Regression after the header-canonicalisation rewrite: the same fixture re-decoded to identical sip_message_count (33) and call_count (8) as before the compact-header change, confirming that Call-ID, From, and To parse identically through canonicalisation.
• Response-level Simplify, Explain, and Evidence were exercised on the cancel-timer capture, both whole-capture and scoped to a single Call-ID. The de-duplicating narrative composer states the salient finding once, always ends on the capture-point caveat, and singularises correctly on the focused single-call view.
• The "possible related call legs" correlation fired correctly on the cancel-timer capture, pairing the two cancelled INVITE legs (inbound and outbound, cancellation evidence within 150ms). This positive case is witnessed on real capture data, confirmed across two separate boxes. The four exclusion gates (overlapping window, opposite direction with shared PBX endpoint, 183 on both legs, sub-250ms cancellation) are reviewed in code, and OPTIONS and other non-INVITE transactions are excluded upstream because they never enter the correlation candidate set. The no-fire path on two genuinely unrelated cancelled INVITEs has not been adversarially reproduced, as available captures did not contain that shape; the gating is therefore analysed and the positive path confirmed, but the negative path rests on code review rather than a witnessed capture.
• The plain-English fact line was checked on the cancel-timer cancelled INVITEs in both the focus picker rows and the call-scoped focus-context block. In the picker, the two legs of the forwarded call render with direction-framed wording, "Outbound leg from 07000000628 to +447000000629 ... cancelled after 26.4s before answer" and "Inbound leg from 07000000628 to +442000000547 ... cancelled ...", each with a single direction prefix and the correct cancelled verb (never "connected"). The call-scoped focus-context line for the same calls keeps the plain "07000000628 tried to call ..." wording, confirming the standalone path is unchanged and the picker/focus-context asymmetry is as intended. Caller and callee were correctly digit-extracted from the From and To URIs throughout, and the OPTIONS row stayed silent. The friendly line's degrade to the URI host, for a From or To whose user part is not a clean number, is now witnessed in rendered output on a separate real capture: a cancelled-leg capture rendered "Outbound leg from 192.168.9.87 to +447000000628" and "Inbound leg from 203.0.113.44 to +442000000060", where the originating side carried a non-numeric user part and the line fell back to the URI host while the numeric destination kept its number. The digit-check gate holds in both directions: on the same and other captures, numeric users such as 2001, +442000000666, and +441000000521 took the number branch and were not replaced by a host. The host fallback prints the URI host verbatim as a fact about the signalling; it does not assert whether the endpoint is internal or external, nor ascribe a PBX or trunk role, which the SIP transaction alone does not prove.
• The plain-English "This means:" gloss on response-level Simplify was witnessed on both 2025_05_22 answered captures, whole-capture and scoped to the answered Call-ID. The gloss renders on its own line below the existing Simplify text. Both captures are answered with RTP in both directions, and the gloss correctly read "at least one call was answered, and we saw audio going both ways at this capture point", confirming the RTP-aware answered branch that reads the answered RTP status to describe media visibility. Testing also exercised the salient-finding priority: an answered call whose talk-time shows up as a multi-second signalling gap initially glossed as "there was a long pause", because the gap outranked answered in the shared salient selector. The selector now ranks answered above the gap, so a benign gap on a cleanly-completed call no longer becomes either the headline sentence or the gloss; the headline reads as an answered 2xx and the gloss as the answered line, with every other Simplify sentence (totals, unparsed-payload line, caveat) byte-identical before and after. The cancelled gloss ("the call ended before it was answered") was witnessed on the cancel-timer capture across two boxes. Explain was confirmed untouched, still surfacing the gap context in its fuller register. The remaining gloss variants (busy, one-way RTP and RTP-absent on answered calls, and no-INVITE) are reasoned and code-covered but not yet witnessed, as no available capture triggers them.
• php -l Tools/PcapAnalysis.php and php -l Frogman.class.php clean.
• Captures exercised across a wider real-world set on a FreePBX 17 box: OPTIONS-only captures, REGISTER transactions, authentication challenges, answered, busy, failed, and incomplete-capture flows, RTP present and absent, focused Call-ID analysis, the Simplify/Explain/Evidence actions, captures with more than two INVITE call flows, captures near the size limit, and captures with malformed or unusual caller identifiers. A capture decoding to no INVITE call flows renders the focus picker with an explicit "Calls found / None. No INVITE call flows were decoded in this capture." rather than an empty section. The command-chip safety guard was confirmed with an adversarial test|evil.pcap filename: the file and path remained visible, no clickable command was emitted, and normal Sysadmin captures remained clickable.
• fm_list_pcaps was checked for newest-first ordering with Sysadmin captures sorted by the timestamp parsed from the sysadmin_YYYY_MM_DD_HH_MM_SS filename (falling back to filemtime() when the name does not parse), the displayed timestamp matching the one used for sorting, deduplication by resolved path, and strict limit validation. php -l Tools/ListPcaps.php clean.
• Focused single-Call-ID views were confirmed to label their scope correctly ("This focused analysis...", "This focused view...") and to present capture-wide warnings, including the TCP-reassembly cap warning, as capture warnings that apply elsewhere in the capture rather than to the focused call. The SIP message output cap is honoured across both UDP and reassembled-TCP parsing, so a mixed-transport capture cannot exceed the configured cap, and the cap warning is not duplicated.
• The per-stream TCP reassembly cap was corrected so rejected payloads no longer increment the stored-byte counter, and the per-stream warning is emitted once rather than once per skipped packet. php -l Tools/PcapAnalysis.php and full repository PHP lint clean. This path is confirmed by code review and uses the same deduplication pattern as the witnessed SIP message cap, but was not exercised against a live capture because the available SysAdmin Pro captures are UDP-only and therefore do not enter the SIP-over-TCP reassembly path.

What's deliberately not in this PR

• pcapng. Classic libpcap only; pcapng is detected and returns a "convert with editcap first" hint rather than mis-parsing it.
• RTP payload decoding or audio analysis. The tool counts and correlates RTP; it never inspects media content.
• Anything that starts or configures a capture. Capture creation stays in Sysadmin.

Also fixed

Fixed chat scroll positioning for long bot responses. New assistant replies now settle at the start of the response, with Mermaid rendering allowed to complete first and manual user scrolling cancelling the pending auto-scroll.

Files changed

• Tools/PcapAnalysis.php (new)
• Tools/ListPcaps.php (new)
• Frogman.class.php (formatter cases for fm_analyze_pcap and fm_list_pcaps)
• Tools/ChatParser.php (routing patterns for analyse/analyze and the pcap-action chips)
• assets/js/chat.js (scroll-to-start behaviour for bot responses; see Also fixed)

[mwtcmi]

Kieran,

Really nice work on this. The safety posture is exactly right for a binary parser touching an admin-permission surface: NUL check, extension-then-realpath-then-extension-again on the path, allowlist prefix with trailing separator, pcapng magic detected and refused cleanly. The command-chip guard degrading to plain text on hostile filenames is the same pattern as the GHSA-7qvv formatter fix, and it's in every relevant place.

The evidence discipline shows in the output too. "No RTP seen at this capture point" over "no audio," leg-framed wording only in the picker where the sibling leg is visible, related-legs surfaced only when four independent gates agree. These are decisions I'd expect from someone who reads a lot of captures. It reads well.

Two things to sort before merge:

1. Tools/ListPcaps.php indentation is broken. From public function description() onward, every method is indented one level deeper than the last. Syntactically fine but unreadable. Please reformat, either by hand or with php-cs-fixer.

2. Split the assets/js/chat.js scroll fix into a separate PR. The change itself looks reasonable (scroll bot messages to start, wait for Mermaid, cancel on user input) but it's unrelated to the pcap feature. I'd like to review and ship it independently so it can be reverted on its own if it regresses something in chat rendering.

One soft ask, take it or leave it:

3. Adversarial-capture pass. The test plan is thorough on real captures but doesn't mention deliberately malformed ones. A truncated header, a bad IP length field, an oversized SDP. The bounds discipline should hold, but this is where you'd want to confirm before shipping. Not a blocker if you're comfortable with the tail risk.

Paste-ready version for a Claude Code session:

Please make two changes to the frogman branch for PR #35:

1. Reformat Tools/ListPcaps.php. The current file has broken indentation where every method is indented one level deeper than the previous one. Run php-cs-fixer on it or reformat by hand so the class body reads at a single normal indent level.

2. Split the assets/js/chat.js scroll-to-start change into its own branch and open a separate PR for it. Remove those changes from the PR Add fm_analyze_pcap and fm_list_pcaps: read-only SIP capture decoder with evidence-scoped analysis #35 branch so it contains only the pcap decoder, list tool, formatter cases, and chat parser routes.

Force-push the cleaned-up branch to kier_dev.

Once those are in, I'll verify on the dev box and ship as v2.9.0 (feature addition, tool count 255 → 257), crediting you in the tag annotation. Thanks for a solid contribution.

[kierknoby]

Thanks, Mike. I have fixed the ListPcaps.php indentation and removed the unrelated chat.js scroll changes from PR #35. The scroll work has been preserved separately for a follow-up PR.

[kierknoby]

Hi Mike,

The two requested cleanup items are complete, and I also ran the optional adversarial-capture pass.

• Truncated capture: tested both an incomplete trailing packet-record header and a packet whose declared captured length exceeded the remaining file bytes. Both failed safely. The latter emitted Packet 7629 is truncated in the capture file and returned the remaining analysis normally.
• Bad IPv4 length field: set the first IPv4 total-length field to 65535 and verified the value on the test box. Frogman returned structured output with no exception and no bogus SIP analysis.
• Oversized SDP: tested a deliberately oversized SIP/SDP packet. Frogman rejected it at the per-packet safety cap with Packet 1 exceeded per-packet safety cap.

No crashes, PHP errors, runaway output or excessive memory use were observed, and no further code changes were required.

The PR is ready for your final verification and merge.

Thanks,
Kieran