ai-assistant: Add shared packages, skills, and proactive diagnosis

[illume]

Summary

This PR separates the AI Assistant into shared runtime and UI packages, adds a standalone CLI, and reduces the Headlamp plugin to its host-specific integrations. The result is less duplicated code, clearer ownership boundaries, broader test coverage, and shared behavior across the UI and command line.

Enhancements

• Skills: Adds configurable skills from GitHub sources with loading, inspection, keyword or embedding routing, and prompt injection, so deployments can extend the assistant with domain-specific guidance without changing application code.
• Provider auto-detection: Adds GitHub Copilot support and automatic discovery of Copilot, Azure OpenAI, and Ollama configurations, so users can review and save available providers instead of entering every setting manually.
• Proactive diagnosis: Automatically starts investigating critical Kubernetes events as they occur, so when users open the assistant the investigation is already underway instead of beginning from manually reconstructed event context.
• Holmes agent integration: Verifies that HolmesGPT is reachable before entering agent mode and, when unavailable, preserves the pending prompt while offering installation, connection-settings, and retry guidance, so users can recover without losing their question or entering a non-working session.
• Cluster-aware MCP: Synchronizes desktop MCP servers with active-cluster changes, so servers using HEADLAMP_CURRENT_CLUSTER can restart with the correct Kubernetes context.
• Offline development mode: Adds fixture-backed models, scripted agents, mock skills, and mock Kubernetes tools, so developers can demonstrate and exercise complete conversations without credentials, network access, or a live cluster.
• Standalone CLI for testing: Adds a headlamp-ai CLI with one-shot, stdin, and interactive modes, primarily so the shared assistant core, providers, skills, and Kubernetes tools can be tested and debugged outside the Headlamp UI.
• Internationalization: Internationalizes 39 production UI files and routes 534 unique English keys through i18next, so the interface is prepared for locale catalogs while fixing interpolation in the separately bundled i18next instance. Translated catalogs are follow-up work.

Skills

Adds configurable skills from GitHub sources with loading, inspection, keyword or embedding routing, and prompt injection, so deployments can extend the assistant with domain-specific guidance without changing application code.

Auto detect provider

So far out of all the different AI providers it auto detects Local llama models, and also GitHub copilot and Azure model foundry ones.

Preview features

Proactive diagnosis

Automatically starts investigating critical Kubernetes events as they occur, so when users open the assistant the investigation is already underway instead of beginning from manually reconstructed event context. Note this is off by default, and can be enabled in the plugin settings.

Accessibility

• 27 improved component modules: Adds substantive accessibility behavior across 27 production ai-ui component modules, so keyboard and assistive-technology users can operate and understand more of the assistant without relying on pointer interaction or visual context.
• Keyboard operation: Adds explicit keyboard behavior to 8 component modules, including a named, focusable AI-panel resize separator with Arrow, Home, and End controls, so users can operate interactive surfaces without dragging or using a mouse.
• Screen-reader context: Adds accessible names or labeling relationships to 26 component modules, live or status announcements to 7, and stronger dialog semantics to 11, so changing progress, contextual actions, and modal purpose are announced more clearly. These categories overlap.
• Dark-theme readability: Adds theme-aware foreground and background colors to rendered code blocks, so code remains readable without forcing a light surface in dark mode.
• Automated accessibility coverage: Adds 56 axe checks across 38 ai-ui test files plus focused keyboard, accessible-name, ARIA-relationship, focus, and dialog tests, so accessibility regressions are caught during component testing. These checks improve coverage but do not claim full WCAG compliance.

Security

• Redacts authorization headers, JWTs, PEM material, AWS access keys, kubeconfig credentials, and common secret fields before formatting MCP output, reducing the chance that credentials are exposed to models or rendered back to users.
• Requires remembered approvals to match every pending tool-call ID and centralizes approve or deny scopes, preventing duplicate or unrelated IDs from accidentally enabling session-wide auto-approval.
• Strictly validates imported MCP server objects, string arguments and environment values, unique names, and boolean approval settings, preventing malformed or ambiguous configurations from being persisted.
• Makes the new CLI Kubernetes tool read-only by default and validates methods, paths, timeouts, and output limits before argument-array execution, reducing mutation risk and command-injection exposure.
• Restricts skill loading protocols and archive paths, rejects traversal and symlinks, and caps file counts and extracted sizes, so untrusted Git or ZIP skill sources cannot freely access the filesystem or exhaust resources.
• Allow-lists provider detection commands, applies execution timeouts, and stores refresh sentinels instead of detected GitHub or Azure tokens, reducing command and credential exposure during automatic setup.

Bug fixes

• Keeps panel visibility session-only instead of restoring it from plugin configuration, so the AI panel no longer reopens unexpectedly after navigation or page reloads.
• Accumulates streamed message chunks and preserves provider-compatible tool-call history, so split tool arguments, multiple tool calls, and OpenAI or Anthropic call/result pairs are not lost or rejected.
• Handles abort signals defensively and recognizes standard abort errors, so cancelling a request reports a clean cancellation instead of racing into a crash or generic failure.
• Tracks user and programmatic scrolling separately while streaming, so new content follows the viewport only when the user has not intentionally scrolled away.
• Assigns stable IDs to saved provider configurations, so reordered settings, duplicate-looking entries, edits, defaults, and deletion of the active provider behave predictably.
• Normalizes MCP settings and legacy tool state, so malformed server definitions are rejected while older tools without an explicit enabled field remain enabled.
• Preserves text surrounding structured log payloads and filters malformed tool output, so useful assistant context is not dropped when rendering logs or partial responses.
• Compares selected clusters independently of array order, so equivalent cluster selections do not trigger unnecessary Kubernetes tool reconfiguration.

Development

• Splits provider, conversation, prompt, skills, MCP, approval, and tool orchestration into @headlamp-k8s/ai-common, so non-UI behavior has a typed, host-independent API that can evolve without Headlamp or React coupling.
• Moves reusable assistant, chat, settings, diagnosis, and output components into @headlamp-k8s/ai-ui, so the Headlamp plugin only supplies cluster APIs, persistence, registration, and component slots instead of maintaining duplicate UI logic.
• Enforces package dependency direction and explicit exports, so accidental React, MUI, or Headlamp imports cannot leak into the common runtime and consumers only rely on supported entry points.
• Centralizes provider construction, tool execution, approvals, MCP processing, and response policies behind focused modules, so provider and LangChain changes have a smaller, more reviewable blast radius.

Testing

• Expands the committed AI Assistant test suite from 1 to 133 test/spec files and from 5 to 34,907 lines of test source, so the shared runtime, UI, CLI, and Headlamp wiring have broad regression coverage instead of a single five-line test harness.
• Achieves 92.93% line / 84.13% branch coverage in ai-common and 96.99% line / 87.84% branch coverage in ai-ui, so the two extracted packages have measurable protection across core logic and UI behavior. These are separate package-level measurements because ai-common uses Istanbul and ai-ui uses V8; one newly added UI file remains below the per-file 80% gate.
• Adds 147 named UI states across 37 Storybook component files, so reviewers and developers can inspect normal, loading, error, empty, configured, and interactive states without manually recreating each scenario.
• Adds fixture chat models, scripted agents, and mock approval, skill, tool, and Holmes implementations, so tests are deterministic and cover complete workflows rather than only isolated rendering.
• Adds a new npm run check script that runs formatting checks, linting, TypeScript validation, and Vitest, so contributors and CI have one consistent command for catching code-quality, type, and regression failures before submission.
• All five checks currently reported for the pushed PR head pass, including the Node 22 AI Assistant build, DCO, and Snyk license and security checks.

Assisted by co pilot and ai-assistant.

[illume]

• folded those extra fix commits into the previous commits where the issue was introduced
• fixed the co-authored-by and signed-off-by for each commit
• made sure each commit passes "npm run check"

[illume]

The 3 big commits are now 9 atomic commits (70 → 76 total):

Skills → 2 commits:

adf95d8 ai-common/skills: Add skill parsing and routing with tests
88892fe ai-common/skills: Add skill loading, I/O adapters, and config with tests

Settings → 5 commits:

e04b8f6 Add TermsDialog and ModelSelector with tests
683a8b7 Add MCP configuration UI
6ad6608 Add skills configuration UI with tests
c802897 Add tools, provider detection, and developer settings with tests
1b28e5c Add SettingsPage with tests

Delete → 2 commits:

1008238 Delete source files superseded by @headlamp-k8s/ai-common
7e5cfeb Delete source files superseded by @headlamp-k8s/ai-ui

Also fixed the pre-existing ordering bug: PromptWidthContext is now committed before AIPanelComponent (which imports it), so the TypeScript check passes at every commit.

[illume]

Split up the docs commits so they are smaller and more atomic.
5 docs commits → 12

[illume]

The docs commits were removed from here, and put into separate PRs.

[illume]

Added review fixes from part 1

• ai-assistant: Add headlamp-k8s/ai-common package — core AI, MCP, and LangChain infrastructure #891

[illume]

In #891 copilot kept finding more and more issues, I would fix and it would report more. 100s found and fixed so far (with tests).

Instead I moved to creating more tests, and finding bugs that way locally first. It seems to be a faster way of finding the issues before the review agent runs. The test coverage was around 60% before, and I'll try to get it up to 80%. With some more mock testing things which can be enabled like a mock tool stuff (execution and approvals for example). You can use the mock tools, approvals, and skills from the cli and plugin in developer mode too (which is good for manual and e2e testing).

Along the way it found a bunch of user facing issues and some security issues as well... not just edge case error handling bugs. Like sending private keys to AI providers (now redacted) and also issues with tool selection approval bypass and writable tools selection bugs, tools, skills and mcp bugs. The process also helped find some unused code which could be removed.

[ashu8912]

LGTM!!